00010000:00010000:0.0F:1761312676.897630:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0F:1761312677.923373:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761312678.626314:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761312678.626322:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761312678.626326:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761312678.626336:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 921 00000100:00080000:0.0:1761312678.626341:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 11s 00000100:00080000:0.0:1761312678.626347:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761312678.626359:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761312678.626376:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e00b5e9d x1846869307409216/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761312678.626405:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761312678.626408:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0F:1761312678.626504:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307409216:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761312678.626573:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307409024:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761312678.627760:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307409152:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761312678.627822:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761312678.627828:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307409088:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761312678.631532:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761312678.631545:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761312678.631555:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307409216:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761312678.631682:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307409088:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761312678.631694:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307409024:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761312678.631703:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307409152:192.168.203.151@tcp:400:kworker/u8:3.0 00010000:00010000:0.0:1761312678.631730:0:2263:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff951bc4d8c800 lock: 000000009fe8324e/0x8add5bec8ef16c95 lrc: 6/2,0 mode: --/PR res: [0x3f:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 20210 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1761312678.631744:0:2263:0:(ldlm_request.c:641:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-OST0000-osc-ffff951bc4d8c800 lock: 000000009fe8324e/0x8add5bec8ef16c95 lrc: 6/2,0 mode: --/PR res: [0x3f:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 20210 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1761312678.631751:0:2263:0:(ldlm_request.c:562:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-OST0000-osc-ffff951bc4d8c800 lock: 000000009fe8324e/0x8add5bec8ef16c95 lrc: 6/2,0 mode: --/PR res: [0x3f:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x206400000000 nid: local remote: 0x0 expref: -99 pid: 20210 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1761312678.631788:0:2263:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff951bc4d8c800 lock: 000000009fe8324e/0x8add5bec8ef16c95 lrc: 6/2,0 mode: --/PR res: [0x3f:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10206400000000 nid: local remote: 0x0 expref: -99 pid: 20210 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1761312678.631793:0:2263:0:(ldlm_lock.c:909:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0000-osc-ffff951bc4d8c800 lock: 000000009fe8324e/0x8add5bec8ef16c95 lrc: 5/1,0 mode: --/PR res: [0x3f:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10206400000000 nid: local remote: 0x0 expref: -99 pid: 20210 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1761312678.634865:0:2263:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff951bc4d8c800 lock: 000000009fe8324e/0x8add5bec8ef16c95 lrc: 4/1,0 mode: --/PR res: [0x3f:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10206400000000 nid: local remote: 0x0 expref: -99 pid: 20210 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1761312678.634885:0:2263:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-OST0000-osc-ffff951bc4d8c800 lock: 000000009fe8324e/0x8add5bec8ef16c95 lrc: 3/0,0 mode: --/PR res: [0x3f:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10206400000000 nid: local remote: 0x0 expref: -99 pid: 20210 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1761312678.634893:0:2263:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-OST0000-osc-ffff951bc4d8c800 lock: 000000009fe8324e/0x8add5bec8ef16c95 lrc: 4/0,0 mode: --/PR res: [0x3f:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x206400000000 nid: local remote: 0x0 expref: -99 pid: 20210 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1761312678.634904:0:2263:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000009fe8324e already unused, calling callback (00000000ec6e4fdf) 00010000:00010000:0.0:1761312678.634908:0:2263:0:(ldlm_request.c:1232:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0000-osc-ffff951bc4d8c800 lock: 000000009fe8324e/0x8add5bec8ef16c95 lrc: 5/0,0 mode: --/PR res: [0x3f:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x20e400000000 nid: local remote: 0x0 expref: -99 pid: 20210 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1761312678.634922:0:2263:0:(ldlm_request.c:1248:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1761312678.634928:0:2263:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-OST0000-osc-ffff951bc4d8c800 lock: 000000009fe8324e/0x8add5bec8ef16c95 lrc: 3/0,0 mode: --/PR res: [0x3f:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x4a0f400000000 nid: local remote: 0x0 expref: -99 pid: 20210 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1761312678.634934:0:2263:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0000-osc-ffff951bc4d8c800 lock: 000000009fe8324e/0x8add5bec8ef16c95 lrc: 0/0,0 mode: --/PR res: [0x3f:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x4a0f400000000 nid: local remote: 0x0 expref: -99 pid: 20210 timeout: 0 lvb_type: 1 00000100:00100000:0.0:1761312678.634989:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:0:1846869307408960:192.168.203.151@tcp:101:stat.0 00000020:00010000:1.0:1761312678.636367:0:20210:0:(cl_lock.c:53:__cl_lock_trace()) release lock: 00000000ce624980 (00000000958ce16f/0) at cl_lock_release():234 00000020:00010000:1.0:1761312678.636374:0:20210:0:(cl_lock.c:53:__cl_lock_trace()) cancel lock: 00000000ce624980 (00000000958ce16f/0) at cl_lock_cancel():151 00000020:00010000:1.0:1761312678.636380:0:20210:0:(cl_lock.c:53:__cl_lock_trace()) cancel lock: 000000002903e376 (000000001bc5fcd4/1) at cl_lock_cancel():151 00000020:00010000:1.0:1761312678.636383:0:20210:0:(cl_lock.c:53:__cl_lock_trace()) destroy lock: 00000000ce624980 (00000000958ce16f/0) at cl_lock_fini():84 00000020:00010000:1.0:1761312678.636385:0:20210:0:(cl_lock.c:53:__cl_lock_trace()) destroy lock: 000000002903e376 (00000000958ce16f/1) at cl_lock_fini():84 00000080:00200000:1.0:1761312678.636401:0:20210:0:(vvp_io.c:313:vvp_io_fini()) [0x200000402:0x31:0x0] ignore/verify layout 0/1, layout version 2 need write layout 0, restore needed 0 00000080:00200000:1.0:1761312678.636406:0:20210:0:(vvp_io.c:1820:vvp_io_init()) [0x200000402:0x31:0x0] ignore/verify layout 0/1, layout version 2 restore needed 0 00000080:00200000:1.0:1761312678.636414:0:20210:0:(vvp_io.c:313:vvp_io_fini()) [0x200000402:0x31:0x0] ignore/verify layout 0/1, layout version 2 need write layout 0, restore needed 0 00010000:00010000:2.0F:1761312678.947146:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000080:2.0:1761312679.693994:0:20795:0:(module.c:221:libcfs_ioctl()) libcfs ioctl cmd 3221775648 00000001:02000400:2.0:1761312679.694000:0:20795:0:(debug.c:702:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-flr test 32: data should be mirrored to newly created mirror ========================================================== 09:31:18 (1761312678) 00000080:00200000:1.0:1761312679.796500:0:20805:0:(file.c:5227:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](000000004093dac1),name=/ 00000002:00010000:1.0:1761312679.796529:0:20805:0:(mdc_locks.c:1328:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: lookup flags 0 00010000:00010000:1.0:1761312679.796547:0:20805:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 00000000c98a1eff/0x8add5bec8ef16c5d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xe94cf4d911c7edf6 expref: -99 pid: 19118 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1761312679.796561:0:20805:0:(ldlm_lock.c:1494:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 00000000c98a1eff/0x8add5bec8ef16c5d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xe94cf4d911c7edf6 expref: -99 pid: 19118 timeout: 0 lvb_type: 0 00000080:00010000:1.0:1761312679.796572:0:20805:0:(dcache.c:293:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](000000004093dac1) 00000080:00010000:1.0:1761312679.796575:0:20805:0:(llite_internal.h:1711:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](000000004093dac1) for lock 0x8add5bec8ef16c5d 00000080:00010000:1.0:1761312679.796581:0:20805:0:(dcache.c:193:ll_intent_drop_lock()) releasing lock with cookie 0x8add5bec8ef16c5d from it 0000000059652a30 00010000:00010000:1.0:1761312679.796584:0:20805:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 00000000c98a1eff/0x8add5bec8ef16c5d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xe94cf4d911c7edf6 expref: -99 pid: 19118 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1761312679.796602:0:20805:0:(ldlm_lock.c:902:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 00000000c98a1eff/0x8add5bec8ef16c5d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xe94cf4d911c7edf6 expref: -99 pid: 19118 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1761312679.796615:0:20805:0:(file.c:5661:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](000000004093dac1), inode mode 41ed mask 1 00000080:00200000:1.0:1761312679.796629:0:20805:0:(namei.c:1111:ll_lookup_nd()) VFS Op:name=f32.sanity-flr, dir=[0x200000007:0x1:0x0](000000004093dac1), flags=8 00000080:00200000:1.0:1761312679.796634:0:20805:0:(namei.c:833:ll_lookup_it()) VFS Op:name=f32.sanity-flr, dir=[0x200000007:0x1:0x0](000000004093dac1), intent=getattr 00000002:00010000:1.0:1761312679.796895:0:20805:0:(mdc_locks.c:1328:mdc_intent_lock()) (name: f32.sanity-flr,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1761312679.796941:0:20805:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 00000000e72dd7cc/0x8add5bec8ef16c9c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 20805 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1761312679.796950:0:20805:0:(ldlm_request.c:1026:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 00000000e72dd7cc/0x8add5bec8ef16c9c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 20805 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1761312679.796999:0:20805:0:(ldlm_request.c:1114:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 00000000e72dd7cc/0x8add5bec8ef16c9c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 20805 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1761312679.797029:0:20805:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job rm:552c6ffa-be0b-4ca9-86e3-4892e1432c16:20805:1846869307409280:192.168.203.151@tcp:101:rm.0 00000100:00100000:1.0:1761312679.797130:0:20805:0:(client.c:2500:ptlrpc_set_wait()) set 000000001c975cf5 going to sleep for 425 seconds 00000100:00100000:3.0:1761312679.803222:0:20805:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job rm:552c6ffa-be0b-4ca9-86e3-4892e1432c16:20805:1846869307409280:192.168.203.151@tcp:101:rm.0 00010000:00010000:3.0:1761312679.803246:0:20805:0:(ldlm_request.c:641:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 00000000e72dd7cc/0x8add5bec8ef16c9c lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 20805 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761312679.803260:0:20805:0:(ldlm_request.c:562:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 00000000e72dd7cc/0x8add5bec8ef16c9c lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x206400000000 nid: local remote: 0x0 expref: -99 pid: 20805 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761312679.803267:0:20805:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 00000000e72dd7cc/0x8add5bec8ef16c9c lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x10206400000000 nid: local remote: 0x0 expref: -99 pid: 20805 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761312679.803273:0:20805:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 00000000e72dd7cc/0x8add5bec8ef16c9c lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x10206400000000 nid: local remote: 0x0 expref: -99 pid: 20805 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761312679.803280:0:20805:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 00000000e72dd7cc/0x8add5bec8ef16c9c lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x206400000000 nid: local remote: 0x0 expref: -99 pid: 20805 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761312679.803286:0:20805:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000e72dd7cc already unused, calling callback (000000004885965a) 00010000:00010000:3.0:1761312679.803289:0:20805:0:(ldlm_request.c:1546:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 00000000e72dd7cc/0x8add5bec8ef16c9c lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT gid 0 flags: 0x206400000000 nid: local remote: 0x0 expref: -99 pid: 20805 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761312679.803295:0:20805:0:(ldlm_request.c:1556:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 00000000e72dd7cc/0x8add5bec8ef16c9c lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT gid 0 flags: 0x206400000000 nid: local remote: 0x0 expref: -99 pid: 20805 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761312679.803302:0:20805:0:(ldlm_request.c:1232:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 00000000e72dd7cc/0x8add5bec8ef16c9c lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT gid 0 flags: 0x20e400000000 nid: local remote: 0x0 expref: -99 pid: 20805 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761312679.803321:0:20805:0:(ldlm_request.c:1248:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1761312679.803326:0:20805:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 00000000e72dd7cc/0x8add5bec8ef16c9c lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT gid 0 flags: 0x4a0f400000000 nid: local remote: 0x0 expref: -99 pid: 20805 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761312679.803332:0:20805:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 00000000e72dd7cc/0x8add5bec8ef16c9c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT gid 0 flags: 0x4a0f400000000 nid: local remote: 0x0 expref: -99 pid: 20805 timeout: 0 lvb_type: 0 00000002:00100000:3.0:1761312679.803349:0:20805:0:(mdc_locks.c:743:mdc_finish_enqueue()) @@@ op=8 disposition=7, status=0 req@000000009cec3faa x1846869307409280/t0(0) o101->lustre-MDT0000-mdc-ffff951bc4d8c800@192.168.203.151@tcp:12/10 lens 584/640 e 0 to 0 dl 1761313105 ref 1 fl Complete:RQU/0/0 rc 301/301 job:'rm.0' 00010000:00010000:3.0:1761312679.803380:0:20805:0:(ldlm_lock.c:1511:ldlm_lock_match_with_skip()) ### not matched ns 000000003d670472 type 13 mode 30 res 8589934599/1 (0 0) 00000080:00200000:3.0:1761312679.803439:0:20805:0:(file.c:5227:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](000000004093dac1),name=/ 00000002:00010000:3.0:1761312679.803451:0:20805:0:(mdc_locks.c:1328:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: lookup flags 0 00010000:00010000:3.0:1761312679.803459:0:20805:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 00000000c98a1eff/0x8add5bec8ef16c5d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xe94cf4d911c7edf6 expref: -99 pid: 19118 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761312679.803467:0:20805:0:(ldlm_lock.c:1494:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 00000000c98a1eff/0x8add5bec8ef16c5d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xe94cf4d911c7edf6 expref: -99 pid: 19118 timeout: 0 lvb_type: 0 00000080:00010000:3.0:1761312679.803475:0:20805:0:(dcache.c:293:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](000000004093dac1) 00000080:00010000:3.0:1761312679.803478:0:20805:0:(llite_internal.h:1711:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](000000004093dac1) for lock 0x8add5bec8ef16c5d 00000080:00010000:3.0:1761312679.803483:0:20805:0:(dcache.c:193:ll_intent_drop_lock()) releasing lock with cookie 0x8add5bec8ef16c5d from it 00000000cb4a2059 00010000:00010000:3.0:1761312679.803486:0:20805:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 00000000c98a1eff/0x8add5bec8ef16c5d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xe94cf4d911c7edf6 expref: -99 pid: 19118 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761312679.803493:0:20805:0:(ldlm_lock.c:902:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 00000000c98a1eff/0x8add5bec8ef16c5d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xe94cf4d911c7edf6 expref: -99 pid: 19118 timeout: 0 lvb_type: 0 00000080:00200000:3.0:1761312679.803505:0:20805:0:(file.c:5661:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](000000004093dac1), inode mode 41ed mask 1 00000080:00200000:3.0:1761312679.803524:0:20805:0:(namei.c:1111:ll_lookup_nd()) VFS Op:name=f32.sanity-flr, dir=[0x200000007:0x1:0x0](000000004093dac1), flags=0 00000080:00200000:3.0:1761312679.803528:0:20805:0:(namei.c:833:ll_lookup_it()) VFS Op:name=f32.sanity-flr, dir=[0x200000007:0x1:0x0](000000004093dac1), intent=getattr 00000002:00010000:3.0:1761312679.803540:0:20805:0:(mdc_locks.c:1328:mdc_intent_lock()) (name: f32.sanity-flr,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1761312679.803578:0:20805:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000008bc9530e/0x8add5bec8ef16ca3 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 20805 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761312679.803585:0:20805:0:(ldlm_request.c:1026:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000008bc9530e/0x8add5bec8ef16ca3 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 20805 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761312679.803593:0:20805:0:(ldlm_request.c:1114:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000008bc9530e/0x8add5bec8ef16ca3 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 20805 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1761312679.803613:0:20805:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000101f852a pname:cluuid:pid:xid:nid:opc:job rm:552c6ffa-be0b-4ca9-86e3-4892e1432c16:20805:1846869307409344:192.168.203.151@tcp:101:rm.0 00000100:00100000:3.0:1761312679.804183:0:20805:0:(client.c:2500:ptlrpc_set_wait()) set 00000000c93d0a01 going to sleep for 425 seconds 00000100:00100000:3.0:1761312679.808863:0:20805:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000101f852a pname:cluuid:pid:xid:nid:opc:job rm:552c6ffa-be0b-4ca9-86e3-4892e1432c16:20805:1846869307409344:192.168.203.151@tcp:101:rm.0 00010000:00010000:3.0:1761312679.808884:0:20805:0:(ldlm_request.c:641:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000008bc9530e/0x8add5bec8ef16ca3 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 20805 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761312679.808897:0:20805:0:(ldlm_request.c:562:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000008bc9530e/0x8add5bec8ef16ca3 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x206400000000 nid: local remote: 0x0 expref: -99 pid: 20805 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761312679.808903:0:20805:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000008bc9530e/0x8add5bec8ef16ca3 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x10206400000000 nid: local remote: 0x0 expref: -99 pid: 20805 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761312679.808909:0:20805:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000008bc9530e/0x8add5bec8ef16ca3 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x10206400000000 nid: local remote: 0x0 expref: -99 pid: 20805 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761312679.808917:0:20805:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000008bc9530e/0x8add5bec8ef16ca3 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x206400000000 nid: local remote: 0x0 expref: -99 pid: 20805 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761312679.808934:0:20805:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000008bc9530e already unused, calling callback (000000004885965a) 00010000:00010000:3.0:1761312679.808938:0:20805:0:(ldlm_request.c:1546:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000008bc9530e/0x8add5bec8ef16ca3 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT gid 0 flags: 0x206400000000 nid: local remote: 0x0 expref: -99 pid: 20805 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761312679.808944:0:20805:0:(ldlm_request.c:1556:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000008bc9530e/0x8add5bec8ef16ca3 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT gid 0 flags: 0x206400000000 nid: local remote: 0x0 expref: -99 pid: 20805 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761312679.808951:0:20805:0:(ldlm_request.c:1232:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000008bc9530e/0x8add5bec8ef16ca3 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT gid 0 flags: 0x20e400000000 nid: local remote: 0x0 expref: -99 pid: 20805 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761312679.808957:0:20805:0:(ldlm_request.c:1248:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1761312679.808963:0:20805:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000008bc9530e/0x8add5bec8ef16ca3 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT gid 0 flags: 0x4a0f400000000 nid: local remote: 0x0 expref: -99 pid: 20805 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761312679.808970:0:20805:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000008bc9530e/0x8add5bec8ef16ca3 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT gid 0 flags: 0x4a0f400000000 nid: local remote: 0x0 expref: -99 pid: 20805 timeout: 0 lvb_type: 0 00000002:00100000:3.0:1761312679.808987:0:20805:0:(mdc_locks.c:743:mdc_finish_enqueue()) @@@ op=8 disposition=7, status=0 req@00000000101f852a x1846869307409344/t0(0) o101->lustre-MDT0000-mdc-ffff951bc4d8c800@192.168.203.151@tcp:12/10 lens 584/640 e 0 to 0 dl 1761313105 ref 1 fl Complete:RQU/0/0 rc 301/301 job:'rm.0' 00010000:00010000:3.0:1761312679.809020:0:20805:0:(ldlm_lock.c:1511:ldlm_lock_match_with_skip()) ### not matched ns 000000003d670472 type 13 mode 30 res 8589934599/1 (0 0) 00000080:00200000:3.0:1761312679.809080:0:20805:0:(file.c:5227:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](000000004093dac1),name=/ 00000002:00010000:3.0:1761312679.809092:0:20805:0:(mdc_locks.c:1328:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: lookup flags 0 00010000:00010000:3.0:1761312679.809099:0:20805:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 00000000c98a1eff/0x8add5bec8ef16c5d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xe94cf4d911c7edf6 expref: -99 pid: 19118 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761312679.809113:0:20805:0:(ldlm_lock.c:1494:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 00000000c98a1eff/0x8add5bec8ef16c5d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xe94cf4d911c7edf6 expref: -99 pid: 19118 timeout: 0 lvb_type: 0 00000080:00010000:3.0:1761312679.809129:0:20805:0:(dcache.c:293:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](000000004093dac1) 00000080:00010000:3.0:1761312679.809132:0:20805:0:(llite_internal.h:1711:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](000000004093dac1) for lock 0x8add5bec8ef16c5d 00000080:00010000:3.0:1761312679.809137:0:20805:0:(dcache.c:193:ll_intent_drop_lock()) releasing lock with cookie 0x8add5bec8ef16c5d from it 0000000059652a30 00010000:00010000:3.0:1761312679.809139:0:20805:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 00000000c98a1eff/0x8add5bec8ef16c5d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xe94cf4d911c7edf6 expref: -99 pid: 19118 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761312679.809144:0:20805:0:(ldlm_lock.c:902:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 00000000c98a1eff/0x8add5bec8ef16c5d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xe94cf4d911c7edf6 expref: -99 pid: 19118 timeout: 0 lvb_type: 0 00000080:00200000:3.0:1761312679.809154:0:20805:0:(file.c:5661:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](000000004093dac1), inode mode 41ed mask 1 00000080:00200000:3.0:1761312679.809166:0:20805:0:(namei.c:1111:ll_lookup_nd()) VFS Op:name=f32.sanity-flr-2, dir=[0x200000007:0x1:0x0](000000004093dac1), flags=8 00000080:00200000:3.0:1761312679.809170:0:20805:0:(namei.c:833:ll_lookup_it()) VFS Op:name=f32.sanity-flr-2, dir=[0x200000007:0x1:0x0](000000004093dac1), intent=getattr 00000002:00010000:3.0:1761312679.809179:0:20805:0:(mdc_locks.c:1328:mdc_intent_lock()) (name: f32.sanity-flr-2,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1761312679.809211:0:20805:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000000ae29aba/0x8add5bec8ef16caa lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 20805 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761312679.809216:0:20805:0:(ldlm_request.c:1026:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000000ae29aba/0x8add5bec8ef16caa lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 20805 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761312679.809223:0:20805:0:(ldlm_request.c:1114:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000000ae29aba/0x8add5bec8ef16caa lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 20805 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1761312679.809239:0:20805:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000101f852a pname:cluuid:pid:xid:nid:opc:job rm:552c6ffa-be0b-4ca9-86e3-4892e1432c16:20805:1846869307409408:192.168.203.151@tcp:101:rm.0 00000100:00100000:3.0:1761312679.809302:0:20805:0:(client.c:2500:ptlrpc_set_wait()) set 000000006609e194 going to sleep for 425 seconds 00000100:00100000:0.0:1761312679.814800:0:20805:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000101f852a pname:cluuid:pid:xid:nid:opc:job rm:552c6ffa-be0b-4ca9-86e3-4892e1432c16:20805:1846869307409408:192.168.203.151@tcp:101:rm.0 00010000:00010000:0.0:1761312679.814821:0:20805:0:(ldlm_request.c:641:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000000ae29aba/0x8add5bec8ef16caa lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 20805 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1761312679.814835:0:20805:0:(ldlm_request.c:562:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000000ae29aba/0x8add5bec8ef16caa lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x206400000000 nid: local remote: 0x0 expref: -99 pid: 20805 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1761312679.814842:0:20805:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000000ae29aba/0x8add5bec8ef16caa lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x10206400000000 nid: local remote: 0x0 expref: -99 pid: 20805 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1761312679.814848:0:20805:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000000ae29aba/0x8add5bec8ef16caa lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x10206400000000 nid: local remote: 0x0 expref: -99 pid: 20805 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1761312679.814856:0:20805:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000000ae29aba/0x8add5bec8ef16caa lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x206400000000 nid: local remote: 0x0 expref: -99 pid: 20805 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1761312679.814863:0:20805:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000000ae29aba already unused, calling callback (000000004885965a) 00010000:00010000:0.0:1761312679.814866:0:20805:0:(ldlm_request.c:1546:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000000ae29aba/0x8add5bec8ef16caa lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT gid 0 flags: 0x206400000000 nid: local remote: 0x0 expref: -99 pid: 20805 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1761312679.814872:0:20805:0:(ldlm_request.c:1556:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000000ae29aba/0x8add5bec8ef16caa lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT gid 0 flags: 0x206400000000 nid: local remote: 0x0 expref: -99 pid: 20805 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1761312679.814879:0:20805:0:(ldlm_request.c:1232:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000000ae29aba/0x8add5bec8ef16caa lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT gid 0 flags: 0x20e400000000 nid: local remote: 0x0 expref: -99 pid: 20805 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1761312679.814898:0:20805:0:(ldlm_request.c:1248:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1761312679.814904:0:20805:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000000ae29aba/0x8add5bec8ef16caa lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT gid 0 flags: 0x4a0f400000000 nid: local remote: 0x0 expref: -99 pid: 20805 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1761312679.814911:0:20805:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000000ae29aba/0x8add5bec8ef16caa lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT gid 0 flags: 0x4a0f400000000 nid: local remote: 0x0 expref: -99 pid: 20805 timeout: 0 lvb_type: 0 00000002:00100000:0.0:1761312679.814926:0:20805:0:(mdc_locks.c:743:mdc_finish_enqueue()) @@@ op=8 disposition=7, status=0 req@00000000101f852a x1846869307409408/t0(0) o101->lustre-MDT0000-mdc-ffff951bc4d8c800@192.168.203.151@tcp:12/10 lens 592/640 e 0 to 0 dl 1761313105 ref 1 fl Complete:RQU/0/0 rc 301/301 job:'rm.0' 00010000:00010000:0.0:1761312679.814954:0:20805:0:(ldlm_lock.c:1511:ldlm_lock_match_with_skip()) ### not matched ns 000000003d670472 type 13 mode 30 res 8589934599/1 (0 0) 00000080:00200000:0.0:1761312679.814997:0:20805:0:(file.c:5227:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](000000004093dac1),name=/ 00000002:00010000:0.0:1761312679.815007:0:20805:0:(mdc_locks.c:1328:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: lookup flags 0 00010000:00010000:0.0:1761312679.815013:0:20805:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 00000000c98a1eff/0x8add5bec8ef16c5d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xe94cf4d911c7edf6 expref: -99 pid: 19118 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1761312679.815021:0:20805:0:(ldlm_lock.c:1494:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 00000000c98a1eff/0x8add5bec8ef16c5d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xe94cf4d911c7edf6 expref: -99 pid: 19118 timeout: 0 lvb_type: 0 00000080:00010000:0.0:1761312679.815027:0:20805:0:(dcache.c:293:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](000000004093dac1) 00000080:00010000:0.0:1761312679.815030:0:20805:0:(llite_internal.h:1711:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](000000004093dac1) for lock 0x8add5bec8ef16c5d 00000080:00010000:0.0:1761312679.815034:0:20805:0:(dcache.c:193:ll_intent_drop_lock()) releasing lock with cookie 0x8add5bec8ef16c5d from it 00000000cb4a2059 00010000:00010000:0.0:1761312679.815036:0:20805:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 00000000c98a1eff/0x8add5bec8ef16c5d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xe94cf4d911c7edf6 expref: -99 pid: 19118 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1761312679.815043:0:20805:0:(ldlm_lock.c:902:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 00000000c98a1eff/0x8add5bec8ef16c5d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xe94cf4d911c7edf6 expref: -99 pid: 19118 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1761312679.815053:0:20805:0:(file.c:5661:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](000000004093dac1), inode mode 41ed mask 1 00000080:00200000:0.0:1761312679.815072:0:20805:0:(namei.c:1111:ll_lookup_nd()) VFS Op:name=f32.sanity-flr-2, dir=[0x200000007:0x1:0x0](000000004093dac1), flags=0 00000080:00200000:0.0:1761312679.815077:0:20805:0:(namei.c:833:ll_lookup_it()) VFS Op:name=f32.sanity-flr-2, dir=[0x200000007:0x1:0x0](000000004093dac1), intent=getattr 00000002:00010000:0.0:1761312679.815087:0:20805:0:(mdc_locks.c:1328:mdc_intent_lock()) (name: f32.sanity-flr-2,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1761312679.815123:0:20805:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000009fe8324e/0x8add5bec8ef16cb1 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 20805 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1761312679.815129:0:20805:0:(ldlm_request.c:1026:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000009fe8324e/0x8add5bec8ef16cb1 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 20805 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1761312679.815137:0:20805:0:(ldlm_request.c:1114:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000009fe8324e/0x8add5bec8ef16cb1 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 20805 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1761312679.815154:0:20805:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job rm:552c6ffa-be0b-4ca9-86e3-4892e1432c16:20805:1846869307409472:192.168.203.151@tcp:101:rm.0 00000100:00100000:0.0:1761312679.815213:0:20805:0:(client.c:2500:ptlrpc_set_wait()) set 000000001b0c3e3a going to sleep for 425 seconds 00000100:00100000:0.0:1761312679.820162:0:20805:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job rm:552c6ffa-be0b-4ca9-86e3-4892e1432c16:20805:1846869307409472:192.168.203.151@tcp:101:rm.0 00010000:00010000:0.0:1761312679.820182:0:20805:0:(ldlm_request.c:641:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000009fe8324e/0x8add5bec8ef16cb1 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 20805 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1761312679.820196:0:20805:0:(ldlm_request.c:562:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000009fe8324e/0x8add5bec8ef16cb1 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x206400000000 nid: local remote: 0x0 expref: -99 pid: 20805 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1761312679.820203:0:20805:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000009fe8324e/0x8add5bec8ef16cb1 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x10206400000000 nid: local remote: 0x0 expref: -99 pid: 20805 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1761312679.820209:0:20805:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000009fe8324e/0x8add5bec8ef16cb1 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x10206400000000 nid: local remote: 0x0 expref: -99 pid: 20805 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1761312679.820216:0:20805:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000009fe8324e/0x8add5bec8ef16cb1 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x206400000000 nid: local remote: 0x0 expref: -99 pid: 20805 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1761312679.820233:0:20805:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000009fe8324e already unused, calling callback (000000004885965a) 00010000:00010000:0.0:1761312679.820237:0:20805:0:(ldlm_request.c:1546:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000009fe8324e/0x8add5bec8ef16cb1 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT gid 0 flags: 0x206400000000 nid: local remote: 0x0 expref: -99 pid: 20805 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1761312679.820243:0:20805:0:(ldlm_request.c:1556:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000009fe8324e/0x8add5bec8ef16cb1 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT gid 0 flags: 0x206400000000 nid: local remote: 0x0 expref: -99 pid: 20805 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1761312679.820249:0:20805:0:(ldlm_request.c:1232:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000009fe8324e/0x8add5bec8ef16cb1 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT gid 0 flags: 0x20e400000000 nid: local remote: 0x0 expref: -99 pid: 20805 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1761312679.820255:0:20805:0:(ldlm_request.c:1248:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:0.0:1761312679.820260:0:20805:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000009fe8324e/0x8add5bec8ef16cb1 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT gid 0 flags: 0x4a0f400000000 nid: local remote: 0x0 expref: -99 pid: 20805 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1761312679.820266:0:20805:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000009fe8324e/0x8add5bec8ef16cb1 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT gid 0 flags: 0x4a0f400000000 nid: local remote: 0x0 expref: -99 pid: 20805 timeout: 0 lvb_type: 0 00000002:00100000:0.0:1761312679.820282:0:20805:0:(mdc_locks.c:743:mdc_finish_enqueue()) @@@ op=8 disposition=7, status=0 req@00000000b5a39135 x1846869307409472/t0(0) o101->lustre-MDT0000-mdc-ffff951bc4d8c800@192.168.203.151@tcp:12/10 lens 592/640 e 0 to 0 dl 1761313105 ref 1 fl Complete:RQU/0/0 rc 301/301 job:'rm.0' 00010000:00010000:0.0:1761312679.820314:0:20805:0:(ldlm_lock.c:1511:ldlm_lock_match_with_skip()) ### not matched ns 000000003d670472 type 13 mode 30 res 8589934599/1 (0 0) 00000080:00200000:3.0:1761312679.840590:0:20810:0:(file.c:5227:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](000000004093dac1),name=/ 00000002:00010000:3.0:1761312679.840618:0:20810:0:(mdc_locks.c:1328:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: lookup flags 0 00010000:00010000:3.0:1761312679.840668:0:20810:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 00000000c98a1eff/0x8add5bec8ef16c5d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xe94cf4d911c7edf6 expref: -99 pid: 19118 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761312679.840690:0:20810:0:(ldlm_lock.c:1494:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 00000000c98a1eff/0x8add5bec8ef16c5d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xe94cf4d911c7edf6 expref: -99 pid: 19118 timeout: 0 lvb_type: 0 00000080:00010000:3.0:1761312679.840894:0:20810:0:(dcache.c:293:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](000000004093dac1) 00000080:00010000:3.0:1761312679.840897:0:20810:0:(llite_internal.h:1711:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](000000004093dac1) for lock 0x8add5bec8ef16c5d 00000080:00010000:3.0:1761312679.840904:0:20810:0:(dcache.c:193:ll_intent_drop_lock()) releasing lock with cookie 0x8add5bec8ef16c5d from it 000000004f5d63b1 00010000:00010000:3.0:1761312679.840906:0:20810:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 00000000c98a1eff/0x8add5bec8ef16c5d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xe94cf4d911c7edf6 expref: -99 pid: 19118 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761312679.840914:0:20810:0:(ldlm_lock.c:902:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 00000000c98a1eff/0x8add5bec8ef16c5d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xe94cf4d911c7edf6 expref: -99 pid: 19118 timeout: 0 lvb_type: 0 00000080:00200000:3.0:1761312679.840927:0:20810:0:(file.c:5661:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](000000004093dac1), inode mode 41ed mask 1 00000080:00200000:3.0:1761312679.840938:0:20810:0:(namei.c:1111:ll_lookup_nd()) VFS Op:name=f32.sanity-flr, dir=[0x200000007:0x1:0x0](000000004093dac1), flags=8 00000080:00200000:3.0:1761312679.840942:0:20810:0:(namei.c:833:ll_lookup_it()) VFS Op:name=f32.sanity-flr, dir=[0x200000007:0x1:0x0](000000004093dac1), intent=getattr 00000002:00010000:3.0:1761312679.840953:0:20810:0:(mdc_locks.c:1328:mdc_intent_lock()) (name: f32.sanity-flr,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1761312679.840998:0:20810:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000008bc9530e/0x8add5bec8ef16cb8 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 20810 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761312679.841005:0:20810:0:(ldlm_request.c:1026:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000008bc9530e/0x8add5bec8ef16cb8 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 20810 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761312679.841014:0:20810:0:(ldlm_request.c:1114:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000008bc9530e/0x8add5bec8ef16cb8 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 20810 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1761312679.841042:0:20810:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job lfs:552c6ffa-be0b-4ca9-86e3-4892e1432c16:20810:1846869307409536:192.168.203.151@tcp:101:lfs.0 00000100:00100000:3.0:1761312679.841252:0:20810:0:(client.c:2500:ptlrpc_set_wait()) set 000000002f9317f5 going to sleep for 425 seconds 00000100:00100000:3.0:1761312679.844902:0:20810:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job lfs:552c6ffa-be0b-4ca9-86e3-4892e1432c16:20810:1846869307409536:192.168.203.151@tcp:101:lfs.0 00010000:00010000:3.0:1761312679.844925:0:20810:0:(ldlm_request.c:641:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000008bc9530e/0x8add5bec8ef16cb8 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 20810 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761312679.844941:0:20810:0:(ldlm_request.c:562:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000008bc9530e/0x8add5bec8ef16cb8 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x206400000000 nid: local remote: 0x0 expref: -99 pid: 20810 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761312679.844947:0:20810:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000008bc9530e/0x8add5bec8ef16cb8 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x10206400000000 nid: local remote: 0x0 expref: -99 pid: 20810 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761312679.844953:0:20810:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000008bc9530e/0x8add5bec8ef16cb8 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x10206400000000 nid: local remote: 0x0 expref: -99 pid: 20810 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761312679.844960:0:20810:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000008bc9530e/0x8add5bec8ef16cb8 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x206400000000 nid: local remote: 0x0 expref: -99 pid: 20810 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761312679.844965:0:20810:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000008bc9530e already unused, calling callback (000000004885965a) 00010000:00010000:3.0:1761312679.844968:0:20810:0:(ldlm_request.c:1546:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000008bc9530e/0x8add5bec8ef16cb8 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT gid 0 flags: 0x206400000000 nid: local remote: 0x0 expref: -99 pid: 20810 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761312679.844974:0:20810:0:(ldlm_request.c:1556:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000008bc9530e/0x8add5bec8ef16cb8 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT gid 0 flags: 0x206400000000 nid: local remote: 0x0 expref: -99 pid: 20810 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761312679.844982:0:20810:0:(ldlm_request.c:1232:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000008bc9530e/0x8add5bec8ef16cb8 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT gid 0 flags: 0x20e400000000 nid: local remote: 0x0 expref: -99 pid: 20810 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761312679.844988:0:20810:0:(ldlm_request.c:1248:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1761312679.844992:0:20810:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000008bc9530e/0x8add5bec8ef16cb8 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT gid 0 flags: 0x4a0f400000000 nid: local remote: 0x0 expref: -99 pid: 20810 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761312679.845015:0:20810:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000008bc9530e/0x8add5bec8ef16cb8 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT gid 0 flags: 0x4a0f400000000 nid: local remote: 0x0 expref: -99 pid: 20810 timeout: 0 lvb_type: 0 00000002:00100000:3.0:1761312679.845029:0:20810:0:(mdc_locks.c:743:mdc_finish_enqueue()) @@@ op=8 disposition=7, status=0 req@0000000039277a44 x1846869307409536/t0(0) o101->lustre-MDT0000-mdc-ffff951bc4d8c800@192.168.203.151@tcp:12/10 lens 584/640 e 0 to 0 dl 1761313105 ref 1 fl Complete:RQU/0/0 rc 301/301 job:'lfs.0' 00010000:00010000:3.0:1761312679.845059:0:20810:0:(ldlm_lock.c:1511:ldlm_lock_match_with_skip()) ### not matched ns 000000003d670472 type 13 mode 30 res 8589934599/1 (0 0) 00000080:00200000:3.0:1761312679.845125:0:20810:0:(file.c:5227:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](000000004093dac1),name=/ 00000002:00010000:3.0:1761312679.845136:0:20810:0:(mdc_locks.c:1328:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: lookup flags 0 00010000:00010000:3.0:1761312679.845143:0:20810:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 00000000c98a1eff/0x8add5bec8ef16c5d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xe94cf4d911c7edf6 expref: -99 pid: 19118 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761312679.845151:0:20810:0:(ldlm_lock.c:1494:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 00000000c98a1eff/0x8add5bec8ef16c5d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xe94cf4d911c7edf6 expref: -99 pid: 19118 timeout: 0 lvb_type: 0 00000080:00010000:3.0:1761312679.845158:0:20810:0:(dcache.c:293:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](000000004093dac1) 00000080:00010000:3.0:1761312679.845161:0:20810:0:(llite_internal.h:1711:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](000000004093dac1) for lock 0x8add5bec8ef16c5d 00000080:00010000:3.0:1761312679.845165:0:20810:0:(dcache.c:193:ll_intent_drop_lock()) releasing lock with cookie 0x8add5bec8ef16c5d from it 00000000a040a78f 00010000:00010000:3.0:1761312679.845167:0:20810:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 00000000c98a1eff/0x8add5bec8ef16c5d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xe94cf4d911c7edf6 expref: -99 pid: 19118 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761312679.845172:0:20810:0:(ldlm_lock.c:902:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 00000000c98a1eff/0x8add5bec8ef16c5d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xe94cf4d911c7edf6 expref: -99 pid: 19118 timeout: 0 lvb_type: 0 00000080:00200000:3.0:1761312679.845184:0:20810:0:(file.c:5661:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](000000004093dac1), inode mode 41ed mask 1 00000080:00200000:3.0:1761312679.845199:0:20810:0:(file.c:5227:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](000000004093dac1),name=/ 00000002:00010000:3.0:1761312679.845203:0:20810:0:(mdc_locks.c:1328:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: lookup flags 0 00010000:00010000:3.0:1761312679.845208:0:20810:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 00000000c98a1eff/0x8add5bec8ef16c5d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xe94cf4d911c7edf6 expref: -99 pid: 19118 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761312679.845221:0:20810:0:(ldlm_lock.c:1494:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 00000000c98a1eff/0x8add5bec8ef16c5d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xe94cf4d911c7edf6 expref: -99 pid: 19118 timeout: 0 lvb_type: 0 00000080:00010000:3.0:1761312679.845228:0:20810:0:(dcache.c:293:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](000000004093dac1) 00000080:00010000:3.0:1761312679.845230:0:20810:0:(llite_internal.h:1711:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](000000004093dac1) for lock 0x8add5bec8ef16c5d 00000080:00010000:3.0:1761312679.845233:0:20810:0:(dcache.c:193:ll_intent_drop_lock()) releasing lock with cookie 0x8add5bec8ef16c5d from it 00000000fe2601cf 00010000:00010000:3.0:1761312679.845235:0:20810:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 00000000c98a1eff/0x8add5bec8ef16c5d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xe94cf4d911c7edf6 expref: -99 pid: 19118 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761312679.845241:0:20810:0:(ldlm_lock.c:902:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 00000000c98a1eff/0x8add5bec8ef16c5d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xe94cf4d911c7edf6 expref: -99 pid: 19118 timeout: 0 lvb_type: 0 00000080:00200000:3.0:1761312679.845248:0:20810:0:(file.c:5661:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](000000004093dac1), inode mode 41ed mask 3 00000080:00200000:3.0:1761312679.845252:0:20810:0:(namei.c:1183:ll_atomic_open()) VFS Op:name=f32.sanity-flr, dir=[0x200000007:0x1:0x0](000000004093dac1), file 00000000e45e555e, open_flags a141, mode 81b6 opened 0 00000080:00200000:3.0:1761312679.845261:0:20810:0:(namei.c:833:ll_lookup_it()) VFS Op:name=f32.sanity-flr, dir=[0x200000007:0x1:0x0](000000004093dac1), intent=open|creat 00000002:00010000:3.0:1761312679.845278:0:20810:0:(mdc_locks.c:1328:mdc_intent_lock()) (name: f32.sanity-flr,[0x200000402:0x32:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0120502 00010000:00010000:3.0:1761312679.845323:0:20810:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000003a9a248b/0x8add5bec8ef16cbf lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 20810 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761312679.845331:0:20810:0:(ldlm_request.c:1026:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000003a9a248b/0x8add5bec8ef16cbf lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 20810 timeout: 0 lvb_type: 0 00000020:00100000:3.0:1761312679.845343:0:20810:0:(genops.c:2339:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff951bc4d8c800: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:3.0:1761312679.845348:0:20810:0:(client.c:742:ptlrpc_reassign_next_xid()) @@@ reassign xid req@0000000039277a44 x1846869307409664/t0(0) o101->lustre-MDT0000-mdc-ffff951bc4d8c800@192.168.203.151@tcp:12/10 lens 528/67832 e 0 to 0 dl 0 ref 1 fl New:PQU/0/ffffffff rc 0/-1 job:'' 00010000:00010000:3.0:1761312679.845356:0:20810:0:(ldlm_request.c:1114:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000003a9a248b/0x8add5bec8ef16cbf lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 20810 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1761312679.845385:0:20810:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job lfs:552c6ffa-be0b-4ca9-86e3-4892e1432c16:20810:1846869307409664:192.168.203.151@tcp:101:lfs.0 00000100:00100000:3.0:1761312679.845499:0:20810:0:(client.c:2500:ptlrpc_set_wait()) set 000000002f9317f5 going to sleep for 425 seconds 00000100:00100000:3.0:1761312679.849934:0:20810:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job lfs:552c6ffa-be0b-4ca9-86e3-4892e1432c16:20810:1846869307409664:192.168.203.151@tcp:101:lfs.0 00010000:00010000:3.0:1761312679.849961:0:20810:0:(ldlm_request.c:714:ldlm_cli_enqueue_fini()) ### server returned different mode CR ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000003a9a248b/0x8add5bec8ef16cbf lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xe94cf4d911c7ef2a expref: -99 pid: 20810 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761312679.849982:0:20810:0:(ldlm_request.c:729:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000003a9a248b/0x8add5bec8ef16cbf lrc: 4/0,1 mode: --/CR res: [0x200000402:0x32:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xe94cf4d911c7ef2a expref: -99 pid: 20810 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761312679.849988:0:20810:0:(ldlm_request.c:742:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000003a9a248b/0x8add5bec8ef16cbf lrc: 4/0,1 mode: --/CR res: [0x200000402:0x32:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xe94cf4d911c7ef2a expref: -99 pid: 20810 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761312679.849999:0:20810:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000003a9a248b/0x8add5bec8ef16cbf lrc: 4/0,1 mode: CR/CR res: [0x200000402:0x32:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xe94cf4d911c7ef2a expref: -99 pid: 20810 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761312679.850007:0:20810:0:(ldlm_request.c:796:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000003a9a248b/0x8add5bec8ef16cbf lrc: 4/0,1 mode: CR/CR res: [0x200000402:0x32:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xe94cf4d911c7ef2a expref: -99 pid: 20810 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761312679.850018:0:20810:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000003a9a248b/0x8add5bec8ef16cbf lrc: 5/1,1 mode: CR/CR res: [0x200000402:0x32:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xe94cf4d911c7ef2a expref: -99 pid: 20810 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761312679.850025:0:20810:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000003a9a248b/0x8add5bec8ef16cbf lrc: 5/1,1 mode: CR/CR res: [0x200000402:0x32:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xe94cf4d911c7ef2a expref: -99 pid: 20810 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761312679.850031:0:20810:0:(ldlm_lock.c:909:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000003a9a248b/0x8add5bec8ef16cbf lrc: 4/1,0 mode: CR/CR res: [0x200000402:0x32:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xe94cf4d911c7ef2a expref: -99 pid: 20810 timeout: 0 lvb_type: 0 00000002:00100000:3.0:1761312679.850059:0:20810:0:(mdc_locks.c:743:mdc_finish_enqueue()) @@@ op=3 disposition=8000037, status=0 req@0000000039277a44 x1846869307409664/t4294967813(4294967813) o101->lustre-MDT0000-mdc-ffff951bc4d8c800@192.168.203.151@tcp:12/10 lens 528/600 e 0 to 0 dl 1761313105 ref 2 fl Complete:RPQU/4/0 rc 0/0 job:'lfs.0' 00000002:00100000:3.0:1761312679.850070:0:20810:0:(mdc_request.c:833:mdc_set_open_replay_data()) @@@ Set up open replay data req@0000000039277a44 x1846869307409664/t4294967813(4294967813) o101->lustre-MDT0000-mdc-ffff951bc4d8c800@192.168.203.151@tcp:12/10 lens 528/600 e 0 to 0 dl 1761313105 ref 2 fl Complete:RPQU/4/0 rc 0/0 job:'lfs.0' 00000002:00010000:3.0:1761312679.850081:0:20810:0:(mdc_locks.c:1179:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000003a9a248b/0x8add5bec8ef16cbf lrc: 3/1,0 mode: CR/CR res: [0x200000402:0x32:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xe94cf4d911c7ef2a expref: -99 pid: 20810 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761312679.850090:0:20810:0:(ldlm_lock.c:1511:ldlm_lock_match_with_skip()) ### not matched ns 000000003d670472 type 13 mode 16 res 8589935618/50 (0 0) 00000080:00200000:3.0:1761312679.850110:0:20810:0:(namei.c:1412:ll_create_it()) VFS Op:name=f32.sanity-flr, dir=[0x200000007:0x1:0x0](000000004093dac1), intent=open|creat 00000080:00200000:3.0:1761312679.850141:0:20810:0:(llite_lib.c:2838:ll_read_inode2()) VFS Op:inode=[0x200000402:0x32:0x0](000000006477b6c6) 00000080:00200000:3.0:1761312679.850149:0:20810:0:(llite_lib.c:2760:ll_update_inode()) inode=[0x200000402:0x32:0x0], updating i_size 0 00000080:00200000:3.0:1761312679.850182:0:20810:0:(file.c:5870:ll_layout_conf()) [0x200000402:0x32:0x0]: layout version change: 4294967294 -> 4294967295 00000080:00010000:3.0:1761312679.850187:0:20810:0:(namei.c:1379:ll_create_node()) setting l_ast_data to inode [0x200000402:0x32:0x0](000000006477b6c6) 00000080:00010000:3.0:1761312679.850190:0:20810:0:(llite_internal.h:1711:ll_set_lock_data()) setting l_data to inode [0x200000402:0x32:0x0](000000006477b6c6) for lock 0x8add5bec8ef16cbf 00000080:00200000:3.0:1761312679.850203:0:20810:0:(file.c:837:ll_file_open()) VFS Op:inode=[0x200000402:0x32:0x0](000000006477b6c6), flags 120501 00000002:00100000:3.0:1761312679.850220:0:20810:0:(mdc_request.c:833:mdc_set_open_replay_data()) @@@ Set up open replay data req@0000000039277a44 x1846869307409664/t4294967813(4294967813) o101->lustre-MDT0000-mdc-ffff951bc4d8c800@192.168.203.151@tcp:12/10 lens 528/600 e 0 to 0 dl 1761313105 ref 2 fl Complete:RPQU/4/0 rc 0/0 job:'lfs.0' 00000080:00010000:3.0:1761312679.850277:0:20810:0:(dcache.c:193:ll_intent_drop_lock()) releasing lock with cookie 0x8add5bec8ef16cbf from it 0000000093161ff2 00010000:00010000:3.0:1761312679.850281:0:20810:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000003a9a248b/0x8add5bec8ef16cbf lrc: 3/1,0 mode: CR/CR res: [0x200000402:0x32:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0xe94cf4d911c7ef2a expref: -99 pid: 20810 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761312679.850288:0:20810:0:(ldlm_lock.c:902:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000003a9a248b/0x8add5bec8ef16cbf lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x32:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xe94cf4d911c7ef2a expref: -99 pid: 20810 timeout: 0 lvb_type: 0 00000080:00200000:3.0:1761312679.850304:0:20810:0:(file.c:5661:ll_inode_permission()) VFS Op:inode=[0x200000402:0x32:0x0](000000006477b6c6), inode mode 81a4 mask 40 00000080:00200000:3.0:1761312679.850354:0:20810:0:(file.c:5661:ll_inode_permission()) VFS Op:inode=[0x200000402:0x32:0x0](000000006477b6c6), inode mode 81a4 mask 2 00000080:00200000:3.0:1761312679.850375:0:20810:0:(xattr.c:358:ll_xattr_set()) VFS Op:inode=[0x200000402:0x32:0x0](000000006477b6c6), xattr lov 00000002:00010000:3.0:1761312679.850390:0:20810:0:(mdc_locks.c:1328:mdc_intent_lock()) (name: ,[0x200000402:0x32:0x0]) in obj [0x200000007:0x1:0x0], intent: open flags 040000002 00010000:00010000:3.0:1761312679.850432:0:20810:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 0000000011008108/0x8add5bec8ef16cc6 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 20810 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761312679.850439:0:20810:0:(ldlm_request.c:1026:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 0000000011008108/0x8add5bec8ef16cc6 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 20810 timeout: 0 lvb_type: 0 00000020:00100000:3.0:1761312679.850449:0:20810:0:(genops.c:2339:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff951bc4d8c800: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:3.0:1761312679.850454:0:20810:0:(client.c:742:ptlrpc_reassign_next_xid()) @@@ reassign xid req@000000006c8fcfe5 x1846869307409792/t0(0) o101->lustre-MDT0000-mdc-ffff951bc4d8c800@192.168.203.151@tcp:12/10 lens 2216/133368 e 0 to 0 dl 0 ref 1 fl New:PQU/0/ffffffff rc 0/-1 job:'' 00010000:00010000:3.0:1761312679.850460:0:20810:0:(ldlm_request.c:1114:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 0000000011008108/0x8add5bec8ef16cc6 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 20810 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1761312679.850478:0:20810:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job lfs:552c6ffa-be0b-4ca9-86e3-4892e1432c16:20810:1846869307409792:192.168.203.151@tcp:101:lfs.0 00000100:00100000:3.0:1761312679.850622:0:20810:0:(client.c:2500:ptlrpc_set_wait()) set 000000002f9317f5 going to sleep for 425 seconds 00000100:00100000:1.0:1761312679.859056:0:2255:0:(events.c:361:request_in_callback()) peer: 12345-192.168.203.151@tcp (source: 12345-192.168.203.151@tcp) 00000100:00100000:0.0:1761312679.859141:0:5568:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1846869322075456 00000100:00100000:0.0:1761312679.859205:0:5568:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.151@tcp, seq: 330 00000100:00100000:0.0:1761312679.859212:0:5568:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000005113602 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:552c6ffa-be0b-4ca9-86e3-4892e1432c16+5:13140:x1846869322075456:12345-192.168.203.151@tcp:104: 00010000:00010000:0.0:1761312679.859228:0:5568:0:(ldlm_lockd.c:2468:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000003a9a248b/0x8add5bec8ef16cbf lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x32:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x420000000000 nid: local remote: 0xe94cf4d911c7ef2a expref: -99 pid: 20810 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1761312679.859304:0:5568:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00010000:00010000:2.0:1761312679.859317:0:11248:0:(ldlm_lockd.c:2789:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00000100:00100000:0.0:1761312679.859323:0:5568:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000005113602 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:552c6ffa-be0b-4ca9-86e3-4892e1432c16+5:13140:x1846869322075456:12345-192.168.203.151@tcp:104: Request processed in 109us (274us total) trans 0 rc 0/0 00010000:00010000:2.0:1761312679.859324:0:11248:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000003a9a248b/0x8add5bec8ef16cbf lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x32:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x420000000000 nid: local remote: 0xe94cf4d911c7ef2a expref: -99 pid: 20810 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1761312679.859336:0:5568:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.151@tcp, seq: 330 00010000:00010000:2.0:1761312679.859337:0:11248:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000003a9a248b already unused, calling callback (000000004885965a) 00010000:00010000:2.0:1761312679.859343:0:11248:0:(ldlm_request.c:1546:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000003a9a248b/0x8add5bec8ef16cbf lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x32:0x0].0x0 bits 0x9/0x9 rrc: 2 type: IBT gid 0 flags: 0x420400000000 nid: local remote: 0xe94cf4d911c7ef2a expref: -99 pid: 20810 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761312679.859351:0:11248:0:(ldlm_request.c:1556:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000003a9a248b/0x8add5bec8ef16cbf lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x32:0x0].0x0 bits 0x9/0x9 rrc: 2 type: IBT gid 0 flags: 0x420400000000 nid: local remote: 0xe94cf4d911c7ef2a expref: -99 pid: 20810 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761312679.859358:0:11248:0:(ldlm_request.c:1232:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000003a9a248b/0x8add5bec8ef16cbf lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x32:0x0].0x0 bits 0x9/0x9 rrc: 2 type: IBT gid 0 flags: 0x428400000000 nid: local remote: 0xe94cf4d911c7ef2a expref: -99 pid: 20810 timeout: 0 lvb_type: 0 00000080:00200000:2.0:1761312679.859394:0:11248:0:(vvp_object.c:137:vvp_conf_set()) [0x200000402:0x32:0x0]: losing layout lock 00010000:00010000:2.0:1761312679.859444:0:11248:0:(ldlm_request.c:1279:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000003a9a248b/0x8add5bec8ef16cbf lrc: 2/0,0 mode: --/CR res: [0x200000402:0x32:0x0].0x0 bits 0x9/0x9 rrc: 2 type: IBT gid 0 flags: 0x4c29400000020 nid: local remote: 0xe94cf4d911c7ef2a expref: -99 pid: 20810 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761312679.859451:0:11248:0:(ldlm_request.c:1325:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1761312679.859475:0:11248:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000003a9a248b/0x8add5bec8ef16cbf lrc: 1/0,0 mode: --/CR res: [0x200000402:0x32:0x0].0x0 bits 0x9/0x9 rrc: 2 type: IBT gid 0 flags: 0x4c29400000020 nid: local remote: 0xe94cf4d911c7ef2a expref: -99 pid: 20810 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761312679.859481:0:11248:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 000000003a9a248b/0x8add5bec8ef16cbf lrc: 0/0,0 mode: --/CR res: [0x200000402:0x32:0x0].0x0 bits 0x9/0x9 rrc: 2 type: IBT gid 0 flags: 0x4c29400000020 nid: local remote: 0xe94cf4d911c7ef2a expref: -99 pid: 20810 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761312679.859500:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1761312679.859502:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1761312679.859503:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761312679.859550:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ffb03e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307409856:192.168.203.151@tcp:103:ldlm_bl_03.0 00000100:00100000:3.0:1761312679.864773:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ffb03e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307409856:192.168.203.151@tcp:103:ldlm_bl_03.0 00000100:00100000:3.0:1761312679.872502:0:20810:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job lfs:552c6ffa-be0b-4ca9-86e3-4892e1432c16:20810:1846869307409792:192.168.203.151@tcp:101:lfs.0 00010000:00010000:3.0:1761312679.872532:0:20810:0:(ldlm_request.c:641:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 0000000011008108/0x8add5bec8ef16cc6 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 20810 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761312679.872550:0:20810:0:(ldlm_request.c:562:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 0000000011008108/0x8add5bec8ef16cc6 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x206400000000 nid: local remote: 0x0 expref: -99 pid: 20810 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761312679.872556:0:20810:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 0000000011008108/0x8add5bec8ef16cc6 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x10206400000000 nid: local remote: 0x0 expref: -99 pid: 20810 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761312679.872582:0:20810:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 0000000011008108/0x8add5bec8ef16cc6 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x10206400000000 nid: local remote: 0x0 expref: -99 pid: 20810 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761312679.872590:0:20810:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 0000000011008108/0x8add5bec8ef16cc6 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x206400000000 nid: local remote: 0x0 expref: -99 pid: 20810 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761312679.872596:0:20810:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 0000000011008108 already unused, calling callback (000000004885965a) 00010000:00010000:3.0:1761312679.872600:0:20810:0:(ldlm_request.c:1546:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 0000000011008108/0x8add5bec8ef16cc6 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT gid 0 flags: 0x206400000000 nid: local remote: 0x0 expref: -99 pid: 20810 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761312679.872606:0:20810:0:(ldlm_request.c:1556:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 0000000011008108/0x8add5bec8ef16cc6 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT gid 0 flags: 0x206400000000 nid: local remote: 0x0 expref: -99 pid: 20810 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761312679.872613:0:20810:0:(ldlm_request.c:1232:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 0000000011008108/0x8add5bec8ef16cc6 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT gid 0 flags: 0x20e400000000 nid: local remote: 0x0 expref: -99 pid: 20810 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761312679.872619:0:20810:0:(ldlm_request.c:1248:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1761312679.872626:0:20810:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 0000000011008108/0x8add5bec8ef16cc6 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT gid 0 flags: 0x4a0f400000000 nid: local remote: 0x0 expref: -99 pid: 20810 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761312679.872631:0:20810:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 0000000011008108/0x8add5bec8ef16cc6 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT gid 0 flags: 0x4a0f400000000 nid: local remote: 0x0 expref: -99 pid: 20810 timeout: 0 lvb_type: 0 00000002:00100000:3.0:1761312679.872704:0:20810:0:(mdc_locks.c:743:mdc_finish_enqueue()) @@@ op=1 disposition=800002b, status=0 req@000000006c8fcfe5 x1846869307409792/t4294967814(4294967814) o101->lustre-MDT0000-mdc-ffff951bc4d8c800@192.168.203.151@tcp:12/10 lens 2216/736 e 0 to 0 dl 1761313105 ref 2 fl Complete:RPQU/4/0 rc 301/301 job:'lfs.0' 00000002:00100000:3.0:1761312679.872721:0:20810:0:(mdc_request.c:833:mdc_set_open_replay_data()) @@@ Set up open replay data req@000000006c8fcfe5 x1846869307409792/t4294967814(4294967814) o101->lustre-MDT0000-mdc-ffff951bc4d8c800@192.168.203.151@tcp:12/10 lens 2216/736 e 0 to 0 dl 1761313105 ref 2 fl Complete:RPQU/4/0 rc 301/301 job:'lfs.0' 00000080:00200000:2.0:1761312679.872888:0:20810:0:(vvp_io.c:1820:vvp_io_init()) [0x200000402:0x32:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:2.0:1761312679.872904:0:20810:0:(vvp_io.c:313:vvp_io_fini()) [0x200000402:0x32:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0 00000080:00200000:2.0:1761312679.872948:0:20810:0:(llite_lib.c:2760:ll_update_inode()) inode=[0x200000402:0x32:0x0], updating i_size 0 00000002:00100000:2.0:1761312679.872983:0:20810:0:(mdc_request.c:833:mdc_set_open_replay_data()) @@@ Set up open replay data req@000000006c8fcfe5 x1846869307409792/t4294967814(4294967814) o101->lustre-MDT0000-mdc-ffff951bc4d8c800@192.168.203.151@tcp:12/10 lens 656/736 e 0 to 0 dl 1761313105 ref 2 fl Complete:RPQU/4/0 rc 301/301 job:'lfs.0' 00000002:00100000:2.0:1761312679.873011:0:20810:0:(mdc_request.c:944:mdc_close()) @@@ matched open req@000000006c8fcfe5 x1846869307409792/t4294967814(4294967814) o101->lustre-MDT0000-mdc-ffff951bc4d8c800@192.168.203.151@tcp:12/10 lens 656/736 e 0 to 0 dl 1761313105 ref 2 fl Complete:RPQU/4/0 rc 301/301 job:'lfs.0' 00000020:00100000:2.0:1761312679.873036:0:20810:0:(genops.c:2339:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff951bc4d8c800: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:2.0:1761312679.873041:0:20810:0:(client.c:742:ptlrpc_reassign_next_xid()) @@@ reassign xid req@00000000ae09c97d x1846869307409984/t0(0) o35->lustre-MDT0000-mdc-ffff951bc4d8c800@192.168.203.151@tcp:23/10 lens 392/2192 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761312679.873061:0:20810:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ae09c97d pname:cluuid:pid:xid:nid:opc:job lfs:552c6ffa-be0b-4ca9-86e3-4892e1432c16:20810:1846869307409984:192.168.203.151@tcp:35:lfs.0 00000100:00100000:2.0:1761312679.874519:0:20810:0:(client.c:2500:ptlrpc_set_wait()) set 00000000bcf77cda going to sleep for 6 seconds 00000100:00100000:2.0:1761312679.879078:0:20810:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ae09c97d pname:cluuid:pid:xid:nid:opc:job lfs:552c6ffa-be0b-4ca9-86e3-4892e1432c16:20810:1846869307409984:192.168.203.151@tcp:35:lfs.0 00000002:00100000:2.0:1761312679.879109:0:20810:0:(mdc_request.c:854:mdc_free_open()) @@@ free open request, rq_replay=0 req@000000006c8fcfe5 x1846869307409792/t4294967814(4294967814) o101->lustre-MDT0000-mdc-ffff951bc4d8c800@192.168.203.151@tcp:12/10 lens 656/736 e 0 to 0 dl 1761313105 ref 2 fl Complete:RQU/4/0 rc 301/301 job:'lfs.0' 00000080:00200000:2.0:1761312679.879156:0:20810:0:(file.c:410:ll_file_release()) VFS Op:inode=[0x200000402:0x32:0x0](000000006477b6c6) 00000002:00100000:2.0:1761312679.879187:0:20810:0:(mdc_request.c:944:mdc_close()) @@@ matched open req@0000000039277a44 x1846869307409664/t4294967813(4294967813) o101->lustre-MDT0000-mdc-ffff951bc4d8c800@192.168.203.151@tcp:12/10 lens 528/600 e 0 to 0 dl 1761313105 ref 1 fl Complete:RPQU/4/ffffffff rc 0/-1 job:'lfs.0' 00000020:00100000:2.0:1761312679.879210:0:20810:0:(genops.c:2339:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff951bc4d8c800: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:2.0:1761312679.879214:0:20810:0:(client.c:742:ptlrpc_reassign_next_xid()) @@@ reassign xid req@000000008a641e2e x1846869307410112/t0(0) o35->lustre-MDT0000-mdc-ffff951bc4d8c800@192.168.203.151@tcp:23/10 lens 392/2192 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761312679.879231:0:20810:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008a641e2e pname:cluuid:pid:xid:nid:opc:job lfs:552c6ffa-be0b-4ca9-86e3-4892e1432c16:20810:1846869307410112:192.168.203.151@tcp:35:lfs.0 00000100:00100000:2.0:1761312679.879297:0:20810:0:(client.c:2500:ptlrpc_set_wait()) set 00000000bcf77cda going to sleep for 6 seconds 00000100:00100000:2.0:1761312679.889720:0:20810:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008a641e2e pname:cluuid:pid:xid:nid:opc:job lfs:552c6ffa-be0b-4ca9-86e3-4892e1432c16:20810:1846869307410112:192.168.203.151@tcp:35:lfs.0 00000002:00100000:2.0:1761312679.889772:0:20810:0:(mdc_request.c:854:mdc_free_open()) @@@ free open request, rq_replay=0 req@0000000039277a44 x1846869307409664/t4294967813(4294967813) o101->lustre-MDT0000-mdc-ffff951bc4d8c800@192.168.203.151@tcp:12/10 lens 528/600 e 0 to 0 dl 1761313105 ref 1 fl Complete:RQU/4/ffffffff rc 0/-1 job:'lfs.0' 00000080:00200000:2.0:1761312679.900809:0:20811:0:(file.c:5227:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](000000004093dac1),name=/ 00000002:00010000:2.0:1761312679.900838:0:20811:0:(mdc_locks.c:1328:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: lookup flags 0 00010000:00010000:2.0:1761312679.900958:0:20811:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 00000000c98a1eff/0x8add5bec8ef16c5d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xe94cf4d911c7edf6 expref: -99 pid: 19118 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761312679.900974:0:20811:0:(ldlm_lock.c:1494:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 00000000c98a1eff/0x8add5bec8ef16c5d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xe94cf4d911c7edf6 expref: -99 pid: 19118 timeout: 0 lvb_type: 0 00000080:00010000:2.0:1761312679.900987:0:20811:0:(dcache.c:293:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](000000004093dac1) 00000080:00010000:2.0:1761312679.900991:0:20811:0:(llite_internal.h:1711:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](000000004093dac1) for lock 0x8add5bec8ef16c5d 00000080:00010000:2.0:1761312679.900998:0:20811:0:(dcache.c:193:ll_intent_drop_lock()) releasing lock with cookie 0x8add5bec8ef16c5d from it 00000000cf691b28 00010000:00010000:2.0:1761312679.901000:0:20811:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 00000000c98a1eff/0x8add5bec8ef16c5d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xe94cf4d911c7edf6 expref: -99 pid: 19118 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761312679.901008:0:20811:0:(ldlm_lock.c:902:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 00000000c98a1eff/0x8add5bec8ef16c5d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xe94cf4d911c7edf6 expref: -99 pid: 19118 timeout: 0 lvb_type: 0 00000080:00200000:2.0:1761312679.901025:0:20811:0:(file.c:5661:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](000000004093dac1), inode mode 41ed mask 1 00000080:00200000:2.0:1761312679.901040:0:20811:0:(file.c:5227:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](000000004093dac1),name=/ 00000002:00010000:2.0:1761312679.901045:0:20811:0:(mdc_locks.c:1328:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: lookup flags 0 00010000:00010000:2.0:1761312679.901050:0:20811:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 00000000c98a1eff/0x8add5bec8ef16c5d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xe94cf4d911c7edf6 expref: -99 pid: 19118 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761312679.901057:0:20811:0:(ldlm_lock.c:1494:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 00000000c98a1eff/0x8add5bec8ef16c5d lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xe94cf4d911c7edf6 expref: -99 pid: 19118 timeout: 0 lvb_type: 0 00000080:00010000:2.0:1761312679.901064:0:20811:0:(dcache.c:293:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](000000004093dac1) 00000080:00010000:2.0:1761312679.901066:0:20811:0:(llite_internal.h:1711:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](000000004093dac1) for lock 0x8add5bec8ef16c5d 00000080:00010000:2.0:1761312679.901074:0:20811:0:(dcache.c:193:ll_intent_drop_lock()) releasing lock with cookie 0x8add5bec8ef16c5d from it 00000000ad028d76 00010000:00010000:2.0:1761312679.901076:0:20811:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 00000000c98a1eff/0x8add5bec8ef16c5d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xe94cf4d911c7edf6 expref: -99 pid: 19118 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761312679.901082:0:20811:0:(ldlm_lock.c:902:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 00000000c98a1eff/0x8add5bec8ef16c5d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xe94cf4d911c7edf6 expref: -99 pid: 19118 timeout: 0 lvb_type: 0 00000080:00200000:2.0:1761312679.901089:0:20811:0:(file.c:5661:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](000000004093dac1), inode mode 41ed mask 3 00000080:00200000:2.0:1761312679.901093:0:20811:0:(namei.c:1183:ll_atomic_open()) VFS Op:name=f32.sanity-flr, dir=[0x200000007:0x1:0x0](000000004093dac1), file 00000000c4b9db45, open_flags 8241, mode 81b6 opened 0 00000080:00200000:2.0:1761312679.901102:0:20811:0:(namei.c:833:ll_lookup_it()) VFS Op:name=f32.sanity-flr, dir=[0x200000007:0x1:0x0](000000004093dac1), intent=open|creat 00000002:00010000:2.0:1761312679.901118:0:20811:0:(mdc_locks.c:1328:mdc_intent_lock()) (name: f32.sanity-flr,[0x200000402:0x33:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0101102 00010000:00010000:2.0:1761312679.901162:0:20811:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 00000000fc331230/0x8add5bec8ef16ccd lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 20811 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761312679.901168:0:20811:0:(ldlm_request.c:1026:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 00000000fc331230/0x8add5bec8ef16ccd lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 20811 timeout: 0 lvb_type: 0 00000020:00100000:2.0:1761312679.901179:0:20811:0:(genops.c:2339:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff951bc4d8c800: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:2.0:1761312679.901187:0:20811:0:(client.c:742:ptlrpc_reassign_next_xid()) @@@ reassign xid req@0000000081bf2a4e x1846869307410240/t0(0) o101->lustre-MDT0000-mdc-ffff951bc4d8c800@192.168.203.151@tcp:12/10 lens 2224/67832 e 0 to 0 dl 0 ref 1 fl New:PQU/0/ffffffff rc 0/-1 job:'' 00010000:00010000:2.0:1761312679.901572:0:20811:0:(ldlm_request.c:1114:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 00000000fc331230/0x8add5bec8ef16ccd lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 20811 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1761312679.901596:0:20811:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000081bf2a4e pname:cluuid:pid:xid:nid:opc:job dd:552c6ffa-be0b-4ca9-86e3-4892e1432c16:20811:1846869307410240:192.168.203.151@tcp:101:dd.0 00000100:00100000:2.0:1761312679.903624:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 00000000bcf77cda going to sleep for 425 seconds 00000100:00100000:2.0:1761312679.916566:0:20811:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000081bf2a4e pname:cluuid:pid:xid:nid:opc:job dd:552c6ffa-be0b-4ca9-86e3-4892e1432c16:20811:1846869307410240:192.168.203.151@tcp:101:dd.0 00010000:00010000:2.0:1761312679.916596:0:20811:0:(ldlm_request.c:714:ldlm_cli_enqueue_fini()) ### server returned different mode CR ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 00000000fc331230/0x8add5bec8ef16ccd lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xe94cf4d911c7ef62 expref: -99 pid: 20811 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761312679.916629:0:20811:0:(ldlm_request.c:729:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 00000000fc331230/0x8add5bec8ef16ccd lrc: 4/0,1 mode: --/CR res: [0x200000402:0x32:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xe94cf4d911c7ef62 expref: -99 pid: 20811 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761312679.916671:0:20811:0:(ldlm_request.c:742:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 00000000fc331230/0x8add5bec8ef16ccd lrc: 4/0,1 mode: --/CR res: [0x200000402:0x32:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xe94cf4d911c7ef62 expref: -99 pid: 20811 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761312679.916681:0:20811:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 00000000fc331230/0x8add5bec8ef16ccd lrc: 4/0,1 mode: CR/CR res: [0x200000402:0x32:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xe94cf4d911c7ef62 expref: -99 pid: 20811 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761312679.916691:0:20811:0:(ldlm_request.c:796:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 00000000fc331230/0x8add5bec8ef16ccd lrc: 4/0,1 mode: CR/CR res: [0x200000402:0x32:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xe94cf4d911c7ef62 expref: -99 pid: 20811 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761312679.916704:0:20811:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 00000000fc331230/0x8add5bec8ef16ccd lrc: 5/1,1 mode: CR/CR res: [0x200000402:0x32:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xe94cf4d911c7ef62 expref: -99 pid: 20811 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761312679.916711:0:20811:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 00000000fc331230/0x8add5bec8ef16ccd lrc: 5/1,1 mode: CR/CR res: [0x200000402:0x32:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xe94cf4d911c7ef62 expref: -99 pid: 20811 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761312679.916718:0:20811:0:(ldlm_lock.c:909:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 00000000fc331230/0x8add5bec8ef16ccd lrc: 4/1,0 mode: CR/CR res: [0x200000402:0x32:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xe94cf4d911c7ef62 expref: -99 pid: 20811 timeout: 0 lvb_type: 0 00000002:00100000:2.0:1761312679.916727:0:20811:0:(mdc_locks.c:743:mdc_finish_enqueue()) @@@ op=3 disposition=2b, status=0 req@0000000081bf2a4e x1846869307410240/t4294967817(4294967817) o101->lustre-MDT0000-mdc-ffff951bc4d8c800@192.168.203.151@tcp:12/10 lens 2224/736 e 0 to 0 dl 1761313105 ref 2 fl Complete:RPQU/4/0 rc 0/0 job:'dd.0' 00000002:00100000:2.0:1761312679.916739:0:20811:0:(mdc_request.c:833:mdc_set_open_replay_data()) @@@ Set up open replay data req@0000000081bf2a4e x1846869307410240/t4294967817(4294967817) o101->lustre-MDT0000-mdc-ffff951bc4d8c800@192.168.203.151@tcp:12/10 lens 2224/736 e 0 to 0 dl 1761313105 ref 2 fl Complete:RPQU/4/0 rc 0/0 job:'dd.0' 00000002:00010000:2.0:1761312679.916750:0:20811:0:(mdc_locks.c:847:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 136 ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 00000000fc331230/0x8add5bec8ef16ccd lrc: 3/1,0 mode: CR/CR res: [0x200000402:0x32:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xe94cf4d911c7ef62 expref: -99 pid: 20811 timeout: 0 lvb_type: 0 00000002:00010000:2.0:1761312679.916789:0:20811:0:(mdc_locks.c:1179:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 00000000fc331230/0x8add5bec8ef16ccd lrc: 3/1,0 mode: CR/CR res: [0x200000402:0x32:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xe94cf4d911c7ef62 expref: -99 pid: 20811 timeout: 0 lvb_type: 3 00010000:00010000:2.0:1761312679.916798:0:20811:0:(ldlm_lock.c:1511:ldlm_lock_match_with_skip()) ### not matched ns 000000003d670472 type 13 mode 16 res 8589935618/50 (0 0) 00000080:00200000:2.0:1761312679.916850:0:20811:0:(namei.c:157:ll_iget()) got inode: [0x200000402:0x32:0x0](000000006477b6c6): rc = 0 00000080:00200000:2.0:1761312679.916863:0:20811:0:(file.c:5870:ll_layout_conf()) [0x200000402:0x32:0x0]: layout version change: 4294967294 -> 1 00000080:00010000:2.0:1761312679.916870:0:20811:0:(llite_internal.h:1711:ll_set_lock_data()) setting l_data to inode [0x200000402:0x32:0x0](000000006477b6c6) for lock 0x8add5bec8ef16ccd 00000080:00010000:2.0:1761312679.916881:0:20811:0:(dcache.c:293:ll_lookup_finish_locks()) setting l_data to inode [0x200000402:0x32:0x0](000000006477b6c6) 00000080:00200000:2.0:1761312679.916899:0:20811:0:(file.c:837:ll_file_open()) VFS Op:inode=[0x200000402:0x32:0x0](000000006477b6c6), flags 101101 00000002:00100000:2.0:1761312679.916917:0:20811:0:(mdc_request.c:833:mdc_set_open_replay_data()) @@@ Set up open replay data req@0000000081bf2a4e x1846869307410240/t4294967817(4294967817) o101->lustre-MDT0000-mdc-ffff951bc4d8c800@192.168.203.151@tcp:12/10 lens 664/736 e 0 to 0 dl 1761313105 ref 2 fl Complete:RPQU/4/0 rc 0/0 job:'dd.0' 00000080:00010000:2.0:1761312679.916987:0:20811:0:(dcache.c:193:ll_intent_drop_lock()) releasing lock with cookie 0x8add5bec8ef16ccd from it 00000000529a042a 00010000:00010000:2.0:1761312679.916992:0:20811:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 00000000fc331230/0x8add5bec8ef16ccd lrc: 3/1,0 mode: CR/CR res: [0x200000402:0x32:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0xe94cf4d911c7ef62 expref: -99 pid: 20811 timeout: 0 lvb_type: 3 00010000:00010000:2.0:1761312679.916999:0:20811:0:(ldlm_lock.c:902:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 00000000fc331230/0x8add5bec8ef16ccd lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x32:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xe94cf4d911c7ef62 expref: -99 pid: 20811 timeout: 0 lvb_type: 3 00000080:00200000:2.0:1761312679.917014:0:20811:0:(file.c:5661:ll_inode_permission()) VFS Op:inode=[0x200000402:0x32:0x0](000000006477b6c6), inode mode 81a4 mask 42 00000002:00010000:2.0:1761312679.917045:0:20811:0:(mdc_locks.c:1328:mdc_intent_lock()) (name: ,[0x0:0x0:0x0]) in obj [0x200000402:0x32:0x0], intent: getxattr flags 0 00010000:00010000:2.0:1761312679.917089:0:20811:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 0000000056078a00/0x8add5bec8ef16cd4 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x32:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 20811 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761312679.917107:0:20811:0:(ldlm_request.c:1026:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 0000000056078a00/0x8add5bec8ef16cd4 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x32:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 20811 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761312679.917115:0:20811:0:(ldlm_request.c:1114:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 0000000056078a00/0x8add5bec8ef16cd4 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x32:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 20811 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1761312679.917142:0:20811:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000015f165e6 pname:cluuid:pid:xid:nid:opc:job dd:552c6ffa-be0b-4ca9-86e3-4892e1432c16:20811:1846869307410304:192.168.203.151@tcp:101:dd.0 00000100:00100000:2.0:1761312679.918407:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 00000000846b6245 going to sleep for 425 seconds 00000100:00100000:2.0:1761312679.924799:0:20811:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000015f165e6 pname:cluuid:pid:xid:nid:opc:job dd:552c6ffa-be0b-4ca9-86e3-4892e1432c16:20811:1846869307410304:192.168.203.151@tcp:101:dd.0 00010000:00010000:2.0:1761312679.924823:0:20811:0:(ldlm_request.c:742:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 0000000056078a00/0x8add5bec8ef16cd4 lrc: 4/1,0 mode: --/PR res: [0x200000402:0x32:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xe94cf4d911c7ef77 expref: -99 pid: 20811 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761312679.924839:0:20811:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 0000000056078a00/0x8add5bec8ef16cd4 lrc: 4/1,0 mode: PR/PR res: [0x200000402:0x32:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xe94cf4d911c7ef77 expref: -99 pid: 20811 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761312679.924849:0:20811:0:(ldlm_request.c:796:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 0000000056078a00/0x8add5bec8ef16cd4 lrc: 4/1,0 mode: PR/PR res: [0x200000402:0x32:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xe94cf4d911c7ef77 expref: -99 pid: 20811 timeout: 0 lvb_type: 0 00000002:00100000:2.0:1761312679.924859:0:20811:0:(mdc_locks.c:743:mdc_finish_enqueue()) @@@ op=80 disposition=0, status=0 req@0000000015f165e6 x1846869307410304/t0(0) o101->lustre-MDT0000-mdc-ffff951bc4d8c800@192.168.203.151@tcp:12/10 lens 568/1208 e 0 to 0 dl 1761313105 ref 1 fl Complete:RQU/0/0 rc 0/0 job:'dd.0' 00000002:00010000:2.0:1761312679.924869:0:20811:0:(mdc_locks.c:1179:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 0000000056078a00/0x8add5bec8ef16cd4 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x32:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xe94cf4d911c7ef77 expref: -99 pid: 20811 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761312679.924880:0:20811:0:(ldlm_lock.c:1511:ldlm_lock_match_with_skip()) ### not matched ns 000000003d670472 type 13 mode 4 res 8589935618/50 (0 0) 00000080:00010000:2.0:1761312679.924906:0:20811:0:(llite_internal.h:1711:ll_set_lock_data()) setting l_data to inode [0x200000402:0x32:0x0](000000006477b6c6) for lock 0x8add5bec8ef16cd4 00000080:00010000:2.0:1761312679.924912:0:20811:0:(dcache.c:193:ll_intent_drop_lock()) releasing lock with cookie 0x8add5bec8ef16cd4 from it 0000000037e5f841 00010000:00010000:2.0:1761312679.924915:0:20811:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 0000000056078a00/0x8add5bec8ef16cd4 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x32:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xe94cf4d911c7ef77 expref: -99 pid: 20811 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761312679.924922:0:20811:0:(ldlm_lock.c:902:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff951bc4d8c800 lock: 0000000056078a00/0x8add5bec8ef16cd4 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x32:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xe94cf4d911c7ef77 expref: -99 pid: 20811 timeout: 0 lvb_type: 0 00000080:00200000:2.0:1761312679.924952:0:20811:0:(llite_lib.c:2191:ll_setattr_raw()) lustre: setattr inode [0x200000402:0x32:0x0](000000006477b6c6) from 0 to 0, valid a068, hsm_import 0 00000020:00100000:2.0:1761312679.925010:0:20811:0:(genops.c:2339:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff951bc4d8c800: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1761312679.925014:0:20811:0:(client.c:742:ptlrpc_reassign_next_xid()) @@@ reassign xid req@0000000015f165e6 x1846869307410432/t0(0) o36->lustre-MDT0000-mdc-ffff951bc4d8c800@192.168.203.151@tcp:12/10 lens 488/512 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761312679.925032:0:20811:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000015f165e6 pname:cluuid:pid:xid:nid:opc:job dd:552c6ffa-be0b-4ca9-86e3-4892e1432c16:20811:1846869307410432:192.168.203.151@tcp:36:dd.0 00000100:00100000:2.0:1761312679.925104:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 425 seconds 00000100:00100000:2.0:1761312679.928363:0:20811:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000015f165e6 pname:cluuid:pid:xid:nid:opc:job dd:552c6ffa-be0b-4ca9-86e3-4892e1432c16:20811:1846869307410432:192.168.203.151@tcp:36:dd.0 00000080:00200000:2.0:1761312679.928418:0:20811:0:(file.c:1543:ll_io_set_mirror()) f32.sanity-flr: desiginated mirror: 0 00000080:00200000:2.0:1761312679.928425:0:20811:0:(vvp_io.c:1820:vvp_io_init()) [0x200000402:0x32:0x0] ignore/verify layout 0/1, layout version 0 restore needed 0 00020000:00200000:2.0:1761312679.928448:0:20811:0:(lov_io.c:825:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1761312679.928464:0:20811:0:(vvp_io.c:222:vvp_io_one_lock_index()) lock: 1 [0, 18446744073709551615] 00000008:00010000:2.0:1761312679.928477:0:20811:0:(lustre_osc.h:181:osc_enq2ldlm_flags()) flags: 4 00000008:00010000:2.0:1761312679.928481:0:20811:0:(osc_lock.c:1241:osc_lock_init()) ### lock 00000000dc045c8a, osc lock 000000003d96f18d, flags 0x80000000 00010000:00010000:2.0:1761312679.928510:0:20811:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-OST0000-osc-ffff951bc4d8c800 lock: 00000000fdf0e5f6/0x8add5bec8ef16cdb lrc: 3/0,1 mode: --/PW res: [0x41:0x0:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 20811 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1761312679.928520:0:20811:0:(ldlm_request.c:1026:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x80000000 ns: lustre-OST0000-osc-ffff951bc4d8c800 lock: 00000000fdf0e5f6/0x8add5bec8ef16cdb lrc: 3/0,1 mode: --/PW res: [0x41:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 20811 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1761312679.928552:0:20811:0:(ldlm_request.c:1114:ldlm_cli_enqueue()) ### sending request ns: lustre-OST0000-osc-ffff951bc4d8c800 lock: 00000000fdf0e5f6/0x8add5bec8ef16cdb lrc: 3/0,1 mode: --/PW res: [0x41:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 20811 timeout: 0 lvb_type: 1 00000100:00080000:2.0:1761312679.928571:0:20811:0:(client.c:1659:ptlrpc_send_new_req()) @@@ req waiting for recovery: (FULL != DISCONN) req@000000004459e48d x1846869307410496/t0(0) o101->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 328/400 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/0/ffffffff rc 0/-1 job:'dd.0' 00000100:00100000:2.0:1761312679.928579:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761312679.971874:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761312680.928771:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312680.994150:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761312681.953757:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312682.016892:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761312682.977071:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761312683.041433:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761312683.745510:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761312683.745522:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761312683.745526:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761312683.745542:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 926 00000100:00080000:0.0:1761312683.745548:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 16s 00000100:00080000:0.0:1761312683.745556:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761312683.745570:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761312683.745571:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307410560:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761312683.745591:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000546b1bfe x1846869307410752/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761312683.745618:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761312683.745623:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761312683.745672:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307410624:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761312683.745675:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307410752:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761312683.746742:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307410688:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761312683.754139:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307410624:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761312683.757455:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307410560:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761312683.760015:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761312683.760029:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761312683.760040:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307410752:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761312683.760172:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307410688:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761312684.001020:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312684.064847:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761312685.024909:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312685.089036:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761312686.048884:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761312686.114029:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761312687.072794:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312687.140262:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761312688.096760:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312688.160805:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761312688.864850:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761312688.864858:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761312688.864861:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761312688.864871:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 931 00000100:00080000:0.0:1761312688.864875:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761312688.864881:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761312688.864891:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761312688.864907:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b5a39135 x1846869307411008/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761312688.864948:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761312688.864951:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761312688.865034:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307410816:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761312688.865122:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307410944:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761312688.865155:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761312688.865161:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307410880:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761312688.865572:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307411008:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761312688.872720:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307410816:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761312688.874378:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307410944:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761312688.877532:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761312688.877547:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761312688.877559:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307411008:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761312688.878218:0:2263:0:(client.c:2868:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff951bc4d8c800: committing for last_committed 4294967818 gen 1 00000100:00100000:1.0:1761312688.878398:0:2263:0:(client.c:2899:ptlrpc_free_committed()) @@@ keeping (FL_REPLAY) req@0000000081bf2a4e x1846869307410240/t4294967817(4294967817) o101->lustre-MDT0000-mdc-ffff951bc4d8c800@192.168.203.151@tcp:12/10 lens 664/736 e 0 to 0 dl 1761313105 ref 1 fl Complete:RPQU/4/ffffffff rc 0/-1 job:'dd.0' 00000100:00100000:1.0:1761312688.878416:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307410880:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761312689.120815:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761312689.186410:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761312690.146274:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312690.208821:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761312691.168773:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312691.232807:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761312692.195378:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761312692.257236:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761312693.217008:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312693.280980:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761312693.985381:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761312693.985440:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761312693.985454:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b3aaddc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307411136:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761312693.985551:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761312693.985556:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761312693.985559:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761312693.985564:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 936 00000100:00080000:2.0:1761312693.985566:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761312693.985571:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000100:00100000:2.0:1761312693.985580:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000011b5c99f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307411200:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761312693.985580:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004e00d332 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307411072:192.168.203.151@tcp:400:kworker/u8:2.0 00000008:00100000:2.0:1761312693.985614:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761312693.985632:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006e3466f5 x1846869307411264/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761312693.985704:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761312693.985707:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761312693.985786:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e3466f5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307411264:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00080000:1.0:1761312693.994472:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:3.0:1761312693.994476:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004e00d332 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307411072:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761312693.994479:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761312693.994497:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e3466f5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307411264:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761312693.994574:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000011b5c99f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307411200:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761312693.994586:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b3aaddc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307411136:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761312694.240771:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312694.304830:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761312695.264781:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761312695.329858:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761312696.288812:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312696.352848:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761312697.315791:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312697.376844:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761312698.337041:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761312698.401283:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761312699.104980:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761312699.104990:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761312699.104994:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761312699.105007:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 942 00000100:00080000:2.0:1761312699.105011:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761312699.105018:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761312699.105030:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761312699.105047:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e662bceb x1846869307411520/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761312699.105071:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761312699.105074:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761312699.105121:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000011b5c99f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307411392:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761312699.105286:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b3aaddc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307411328:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761312699.106548:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002292ab57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307411456:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761312699.107043:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e662bceb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307411520:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761312699.113842:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b3aaddc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307411328:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761312699.114333:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000011b5c99f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307411392:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761312699.127784:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002292ab57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307411456:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761312699.128156:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761312699.128164:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761312699.128169:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e662bceb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307411520:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761312699.360897:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312699.424847:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761312700.386030:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312700.450902:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761312701.409838:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761312701.472937:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761312702.432837:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312702.497316:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761312703.457585:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312703.520848:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761312704.224904:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761312704.224914:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761312704.224918:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761312704.224931:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 947 00000100:00080000:2.0:1761312704.224935:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761312704.224943:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761312704.224956:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761312704.224974:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e662bceb x1846869307411776/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761312704.224999:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761312704.225003:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761312704.225106:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e662bceb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307411776:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761312704.225268:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000011b5c99f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307411584:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761312704.225349:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ee08055d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307411712:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761312704.225386:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761312704.225395:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e1be9a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307411648:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761312704.232080:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e1be9a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307411648:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761312704.232105:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000011b5c99f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307411584:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761312704.232116:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ee08055d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307411712:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761312704.261246:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761312704.261260:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761312704.261272:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e662bceb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307411776:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761312704.480916:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761312704.545118:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761312705.504785:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312705.570604:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761312706.529608:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312706.593816:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761312707.552902:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761312707.617548:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761312708.579971:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312708.640870:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761312709.344903:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761312709.344914:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761312709.344918:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1761312709.344932:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307411840:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761312709.344936:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 952 00000100:00080000:1.0:1761312709.344942:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1761312709.344951:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761312709.344964:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761312709.344983:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a1719d14 x1846869307412032/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761312709.345038:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761312709.345042:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761312709.345046:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307411968:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761312709.345068:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307411904:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761312709.345188:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307412032:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761312709.355320:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307411904:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761312709.355382:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307411840:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761312709.355396:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307411968:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761312709.356518:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761312709.356529:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761312709.356538:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307412032:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761312709.600764:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312709.664827:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761312710.626862:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761312710.689584:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200000:1.0:1761312710.880830:0:38:0:(vvp_io.c:1820:vvp_io_init()) [0x200000402:0x32:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:1.0:1761312710.880848:0:38:0:(lov_io.c:825:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:1.0:1761312710.880870:0:38:0:(vvp_io.c:313:vvp_io_fini()) [0x200000402:0x32:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0 00000100:00100000:2.0:1761312711.648768:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312711.712803:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761312712.672784:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312712.736779:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312713.696890:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761312713.760933:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761312714.465580:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761312714.465588:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761312714.465591:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761312714.465602:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 957 00000100:00080000:2.0:1761312714.465605:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761312714.465609:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761312714.465619:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761312714.465633:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000015f165e6 x1846869307412288/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761312714.465744:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761312714.465748:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761312714.465926:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002292ab57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307412096:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761312714.466952:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000015f165e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307412288:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761312714.467997:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e3466f5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307412224:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761312714.475458:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761312714.475482:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b3aaddc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307412160:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761312714.479991:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761312714.480002:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761312714.480012:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000015f165e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307412288:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761312714.480840:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b3aaddc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307412160:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761312714.480855:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002292ab57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307412096:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761312714.480865:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e3466f5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307412224:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761312714.720772:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312714.784810:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312715.745200:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312715.809202:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312716.768775:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761312716.832844:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312717.792762:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312717.857348:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312718.816772:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312718.881005:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761312719.584845:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761312719.584868:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761312719.584872:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761312719.584882:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 962 00000100:00080000:2.0:1761312719.584886:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761312719.584892:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761312719.584901:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761312719.584917:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000aea3302e x1846869307412544/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761312719.584945:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761312719.584949:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761312719.585190:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008a641e2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307412352:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761312719.585282:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ffb03e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307412480:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761312719.585318:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761312719.585325:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ae09c97d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307412416:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761312719.585428:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aea3302e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307412544:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761312719.590606:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008a641e2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307412352:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761312719.593912:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ae09c97d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307412416:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761312719.593930:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ffb03e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307412480:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761312719.594189:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761312719.594195:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761312719.594213:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aea3302e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307412544:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761312719.840984:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761312719.905848:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312720.864807:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312720.928915:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312721.888816:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312721.956597:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312722.912804:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761312722.979192:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312723.937490:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312724.001464:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761312724.705425:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e9376149 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307412608:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761312724.706889:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761312724.706907:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ba5c8e77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307412672:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761312724.707132:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761312724.707138:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761312724.707143:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761312724.707149:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 967 00000100:00080000:2.0:1761312724.707153:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761312724.707159:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761312724.707172:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761312724.707189:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000eca34e1c x1846869307412800/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761312724.707212:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761312724.707215:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761312724.707289:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039978a4e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307412736:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761312724.707372:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eca34e1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307412800:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761312724.714241:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e9376149 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307412608:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761312724.714384:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761312724.714391:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761312724.714396:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eca34e1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307412800:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761312724.714465:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039978a4e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307412736:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761312724.714480:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ba5c8e77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307412672:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761312724.960791:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312725.025774:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312725.984803:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761312726.054152:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312727.008988:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312727.072996:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312728.033309:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312728.096839:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312729.057758:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761312729.121005:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761312729.824954:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307412864:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761312729.826050:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761312729.826057:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761312729.826060:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761312729.826067:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 972 00000100:00080000:1.0:1761312729.826070:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1761312729.826075:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761312729.826086:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761312729.826114:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ed2f9bca x1846869307413056/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761312729.826141:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761312729.826145:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761312729.826165:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307412992:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761312729.826209:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761312729.826214:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307412928:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761312729.826502:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307413056:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00080000:3.0:1761312729.829396:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761312729.829414:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761312729.829423:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307413056:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761312729.829713:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307412928:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761312729.829724:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307412992:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761312729.829734:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307412864:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761312730.082826:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312730.144788:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312731.105193:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312731.168796:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312732.128757:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761312732.193917:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312733.152827:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312733.216827:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312734.177465:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312734.242344:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761312734.944944:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761312734.944953:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761312734.944956:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761312734.944967:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 977 00000100:00100000:2.0:1761312734.944976:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004e00d332 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307413120:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761312734.944984:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761312734.944991:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761312734.945001:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761312734.945019:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f133bb2e x1846869307413312/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761312734.945044:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761312734.945047:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761312734.945059:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307413248:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761312734.945073:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307413184:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761312734.945177:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f133bb2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307413312:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761312734.948376:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307413248:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761312734.949634:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307413184:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761312734.950029:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761312734.950035:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761312734.950040:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f133bb2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307413312:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761312734.950114:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004e00d332 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307413120:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761312735.203530:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761312735.265422:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312736.225174:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312736.290571:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312737.250157:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312737.313824:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312738.272766:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761312738.336847:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312739.296762:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312739.364136:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761312740.064875:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761312740.064882:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761312740.064885:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761312740.064895:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 983 00000100:00080000:3.0:1761312740.064898:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761312740.064904:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761312740.064914:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761312740.064928:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f133bb2e x1846869307413568/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761312740.064946:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761312740.064949:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761312740.065019:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307413376:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761312740.065110:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004e00d332 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307413504:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761312740.065140:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761312740.065145:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cdcc674d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307413440:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761312740.065313:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f133bb2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307413568:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761312740.067574:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307413376:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761312740.068841:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cdcc674d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307413440:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761312740.068857:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004e00d332 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307413504:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761312740.069005:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761312740.069015:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761312740.069024:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f133bb2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307413568:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761312740.320810:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312740.385336:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312741.344823:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761312741.408881:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312742.368771:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312742.432793:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312743.392778:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312743.456813:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312744.417430:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761312744.480924:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761312745.186507:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761312745.186518:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761312745.186523:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761312745.186543:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 988 00000100:00080000:0.0:1761312745.186547:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761312745.186557:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761312745.186569:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761312745.186589:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000084f3d917 x1846869307413824/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761312745.186615:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761312745.186618:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761312745.186787:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307413824:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761312745.187162:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307413632:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761312745.187207:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307413760:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761312745.187230:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761312745.187235:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307413696:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761312745.190685:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761312745.190700:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761312745.190711:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307413824:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761312745.192269:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307413696:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761312745.192286:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307413760:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761312745.202511:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307413632:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761312745.441361:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312745.504798:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312746.464776:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312746.528777:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312747.488772:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761312747.552904:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312748.517134:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312748.576827:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312749.537233:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312749.600855:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761312750.304964:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761312750.304974:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761312750.304978:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761312750.304993:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 993 00000100:00080000:3.0:1761312750.304997:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761312750.305004:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761312750.305016:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761312750.305047:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000101f852a x1846869307414080/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761312750.305076:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761312750.305080:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761312750.305172:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307413888:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761312750.305302:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307414016:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761312750.305342:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761312750.305348:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307413952:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761312750.305671:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000101f852a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307414080:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761312750.311028:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307414016:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761312750.311073:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307413952:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761312750.311084:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307413888:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761312750.313134:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761312750.313157:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761312750.313168:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000101f852a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307414080:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761312750.561134:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761312750.625850:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312751.584817:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312751.649193:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312752.610487:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312752.676012:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312753.632767:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761312753.701525:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312754.656737:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312754.720789:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761312755.426601:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761312755.426612:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761312755.426617:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761312755.426630:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 998 00000100:00080000:2.0:1761312755.426635:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761312755.426686:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761312755.426700:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761312755.426724:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b4eac89a x1846869307414336/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761312755.426768:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761312755.426772:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761312755.426807:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307414208:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761312755.426823:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b4eac89a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307414336:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761312755.427108:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000039eebc2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307414144:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761312755.429485:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f220504f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307414272:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761312755.434249:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000039eebc2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307414144:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:02020000:1.0:1761312755.435134:0:2262:0:(client.c:1361:ptlrpc_check_status()) 11-0: lustre-OST0000-osc-ffff951bc4d8c800: operation ost_connect to node 192.168.203.151@tcp failed: rc = -19 00000100:00100000:2.0:1761312755.435139:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307414208:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761312755.435186:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f220504f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307414272:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761312755.439611:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761312755.439620:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761312755.439630:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b4eac89a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307414336:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761312755.680787:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312755.744912:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312756.704800:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761312756.769415:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312757.728800:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312757.792768:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312758.753069:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312758.816796:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312759.778521:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761312759.841961:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761312760.545037:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761312760.545044:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761312760.545059:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761312760.545068:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1003 00000100:00080000:3.0:1761312760.545071:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761312760.545076:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761312760.545085:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761312760.545101:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ddd3c50a x1846869307414592/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761312760.545124:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761312760.545127:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761312760.545148:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f597cda6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307414464:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761312760.545206:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307414592:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761312760.545758:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307414400:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761312760.546588:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307414528:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761312760.553796:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307414528:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761312760.553869:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f597cda6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307414464:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761312760.553963:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761312760.553968:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761312760.553972:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307414592:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761312760.557690:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307414400:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761312760.800789:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312760.864911:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312761.825092:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312761.888793:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312762.848735:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761312762.912840:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312763.872977:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312763.936805:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312764.896772:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312764.960796:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761312765.664894:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761312765.664901:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761312765.664904:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761312765.664913:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1008 00000100:00080000:2.0:1761312765.664916:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761312765.664922:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761312765.664946:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761312765.664964:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b2298797 x1846869307414848/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761312765.664991:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761312765.664994:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761312765.665019:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001426be00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307414720:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761312765.665113:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1761312765.665119:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307414656:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761312765.665158:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004592c0a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307414784:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761312765.665274:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b2298797 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307414848:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761312765.668268:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307414656:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761312765.670124:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761312765.670132:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761312765.670140:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b2298797 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307414848:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761312765.670400:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001426be00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307414720:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761312765.670417:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004592c0a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307414784:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761312765.920767:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761312765.986001:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761312766.945374:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312767.008814:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761312767.972443:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312768.032831:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761312768.992767:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761312769.056843:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761312770.016780:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312770.080915:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761312770.784890:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f597cda6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307414912:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761312770.784890:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761312770.784899:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761312770.784903:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761312770.784909:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1013 00000100:00080000:3.0:1761312770.784912:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761312770.784918:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761312770.784928:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761312770.784944:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000626f98db x1846869307415104/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761312770.784967:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761312770.784970:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761312770.784978:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307415040:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761312770.785007:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000009911ad3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307414976:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761312770.785016:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000626f98db pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307415104:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761312770.793086:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f597cda6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307414912:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761312770.794103:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000009911ad3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307414976:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761312770.794199:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761312770.794204:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761312770.794209:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000626f98db pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307415104:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761312770.794272:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307415040:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761312771.040765:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312771.104855:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761312772.066259:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761312772.128872:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761312773.089438:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312773.153806:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761312774.112976:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312774.176798:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761312775.137125:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312775.201281:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761312775.904926:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761312775.904934:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761312775.904937:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761312775.904946:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1018 00000100:00080000:3.0:1761312775.904949:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761312775.904954:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761312775.904963:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761312775.904976:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000038534f0b x1846869307415360/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761312775.904998:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761312775.905001:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761312775.905044:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307415232:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761312775.905053:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000038534f0b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307415360:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761312775.905151:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1761312775.905157:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000009911ad3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307415168:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761312775.905183:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007673f690 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307415296:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761312775.915552:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307415232:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761312775.920371:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000009911ad3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307415168:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761312775.920393:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007673f690 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307415296:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761312775.921142:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761312775.921149:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761312775.921156:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000038534f0b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307415360:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761312776.161112:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312776.226389:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761312777.184834:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312777.248796:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761312778.209674:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312778.272866:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761312779.232828:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312779.297401:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761312780.257534:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312780.320819:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761312781.025603:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1761312781.025610:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d32bed84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307415488:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761312781.025610:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761312781.025613:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761312781.025618:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1023 00000100:00080000:3.0:1761312781.025620:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761312781.025625:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761312781.025634:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761312781.025664:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e6d9e11e x1846869307415616/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761312781.025726:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1761312781.025733:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307415424:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761312781.025741:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307415616:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761312781.025756:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e58cc722 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307415552:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761312781.025838:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761312781.025843:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761312781.031601:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307415424:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761312781.033349:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761312781.033358:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761312781.033365:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307415616:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761312781.033634:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e58cc722 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307415552:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761312781.033677:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d32bed84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307415488:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761312781.280903:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312781.346664:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761312782.305928:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312782.369483:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761312783.330762:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312783.392968:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761312784.352770:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312784.422146:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761312785.376753:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312785.441143:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761312786.144994:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761312786.145014:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761312786.145019:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761312786.145037:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1029 00000100:00080000:2.0:1761312786.145041:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761312786.145049:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761312786.145062:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761312786.145083:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000007136630 x1846869307415872/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761312786.145115:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761312786.145141:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761312786.145176:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001426be00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307415744:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761312786.145315:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1761312786.145322:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004592c0a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307415680:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761312786.145359:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307415808:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761312786.145835:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000007136630 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307415872:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761312786.151958:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004592c0a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307415680:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761312786.151990:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307415808:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761312786.152013:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001426be00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307415744:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761312786.152193:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761312786.152200:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761312786.152205:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000007136630 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307415872:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761312786.401527:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312786.466258:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761312787.424990:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312787.490038:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312788.451250:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312788.512831:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312789.472782:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312789.536844:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312790.496807:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312790.561538:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761312791.264908:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761312791.264917:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761312791.264921:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761312791.264934:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1034 00000100:00080000:2.0:1761312791.264937:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761312791.264944:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761312791.264956:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761312791.265252:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000007136630 x1846869307416128/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761312791.265342:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000007136630 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307416128:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761312791.265436:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001426be00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307415936:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761312791.266370:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1761312791.266478:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004592c0a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307416064:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761312791.266541:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761312791.266548:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307416000:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761312791.266910:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00080000:1.0:1761312791.269502:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761312791.269516:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761312791.269528:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000007136630 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307416128:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761312791.272020:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307416000:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761312791.272048:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001426be00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307415936:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761312791.272059:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004592c0a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307416064:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761312791.520954:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312791.585824:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312792.544801:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312792.608976:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312793.568856:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312793.633860:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312794.593383:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312794.658220:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312795.616756:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312795.682316:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761312796.388628:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761312796.388674:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761312796.388678:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761312796.388700:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1039 00000100:00080000:3.0:1761312796.388704:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761312796.388709:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761312796.388718:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761312796.388734:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000bc14d8e x1846869307416384/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761312796.388759:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761312796.388762:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761312796.388788:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cc6dd8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307416256:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761312796.388878:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1761312796.388885:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f364ec74 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307416192:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761312796.388908:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000041c8b6cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307416320:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761312796.389013:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307416384:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00080000:1.0:1761312796.397021:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761312796.397032:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761312796.397039:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307416384:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761312796.397175:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f364ec74 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307416192:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761312796.397195:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000041c8b6cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307416320:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761312796.397204:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc6dd8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307416256:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761312796.640738:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312796.704835:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312797.665210:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312797.728801:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312798.688762:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312798.753184:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312799.712760:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312799.778277:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312800.736768:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312800.800791:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761312801.506610:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761312801.506621:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761312801.506626:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761312801.506670:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1044 00000100:00080000:3.0:1761312801.506696:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761312801.506706:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761312801.506721:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761312801.506742:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cc6dd8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307416448:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761312801.506751:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000bc14d8e x1846869307416640/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761312801.506786:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761312801.506790:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761312801.506849:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307416640:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761312801.506850:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f364ec74 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307416576:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761312801.506860:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000041c8b6cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307416512:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761312801.513125:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f364ec74 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307416576:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761312801.515135:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000041c8b6cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307416512:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761312801.515521:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761312801.515526:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761312801.515531:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307416640:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761312801.515814:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc6dd8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307416448:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761312801.761390:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312801.826850:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312802.784939:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312802.848865:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312803.808763:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312803.873790:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312804.833230:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312804.897019:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312805.856766:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312805.924345:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761312806.624923:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000041c8b6cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307416704:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761312806.626958:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761312806.626965:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761312806.626968:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761312806.626978:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1049 00000100:00080000:3.0:1761312806.626982:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761312806.626987:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761312806.626997:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761312806.627011:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e6d9e11e x1846869307416896/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761312806.627034:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761312806.627038:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761312806.627128:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307416896:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761312806.627423:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d32bed84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307416768:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761312806.627463:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761312806.627469:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e58cc722 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307416832:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761312806.628138:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000041c8b6cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307416704:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761312806.631016:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761312806.631026:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761312806.631034:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307416896:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761312806.634794:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e58cc722 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307416832:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761312806.634818:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d32bed84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307416768:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761312806.881144:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312806.945164:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312807.904794:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312807.970698:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312808.928885:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312808.993089:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312809.953466:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312810.017041:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312810.976773:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312811.042402:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761312811.745144:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761312811.745152:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761312811.745156:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1761312811.745168:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307416960:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761312811.745168:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1054 00000100:00080000:1.0:1761312811.745172:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761312811.745180:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761312811.745189:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761312811.745206:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a15ef099 x1846869307417152/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761312811.745225:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761312811.745229:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761312811.745242:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307417088:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761312811.745252:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307417024:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761312811.745353:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307417152:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761312811.752695:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307416960:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761312811.754258:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307417024:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761312811.756292:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307417088:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761312811.756327:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761312811.756332:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761312811.756337:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307417152:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761312812.001176:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312812.066677:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312813.025258:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312813.088917:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312814.048767:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312814.113386:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312815.075396:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312815.136879:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312816.096831:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312816.160783:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761312816.865057:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761312816.865091:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761312816.865095:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761312816.865109:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1059 00000100:00080000:1.0:1761312816.865113:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761312816.865120:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761312816.865132:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761312816.865153:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000003871442b x1846869307417408/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1761312816.865227:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307417408:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00080000:1.0:1761312816.866144:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761312816.866154:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761312816.866191:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307417216:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761312816.866211:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307417280:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761312816.866260:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307417344:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761312816.880699:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307417280:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761312816.881707:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307417216:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761312816.884136:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307417344:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761312816.888869:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761312816.888885:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761312816.888900:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307417408:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761312817.120806:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312817.184896:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312818.145222:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312818.210153:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312819.171225:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312819.232819:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312820.192826:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312820.260732:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312821.218792:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312821.304295:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761312821.984998:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307417472:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761312821.989432:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761312821.989440:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761312821.989444:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761312821.989456:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1064 00000100:00080000:1.0:1761312821.989460:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761312821.989485:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761312821.989498:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761312821.989518:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000003871442b x1846869307417664/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761312821.989538:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761312821.989542:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761312821.989846:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307417664:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761312821.991021:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307417536:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761312821.991108:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761312821.991115:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307417600:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761312821.991190:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307417472:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761312822.000551:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307417536:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761312822.003709:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761312822.003723:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761312822.003735:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307417664:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761312822.008124:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307417600:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761312822.240809:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312822.304838:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312823.264806:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312823.331219:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312824.288813:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312824.353558:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312825.313777:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312825.376866:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312826.337614:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312826.400893:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761312827.108118:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761312827.108128:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761312827.108133:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761312827.108147:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1070 00000100:00080000:1.0:1761312827.108151:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761312827.108158:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761312827.108189:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761312827.108211:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000009cec3faa x1846869307417920/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761312827.108517:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761312827.108521:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761312827.108556:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307417792:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761312827.108557:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307417920:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761312827.108824:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1761312827.108833:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307417728:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761312827.108864:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307417856:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761312827.113722:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761312827.113738:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761312827.113751:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307417920:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761312827.114309:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307417856:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761312827.114325:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307417792:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761312827.115052:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307417728:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761312827.360932:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312827.425062:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312828.384834:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312828.450863:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312829.408838:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312829.473273:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312830.433489:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312830.497279:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312831.456850:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312831.523005:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761312832.224860:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761312832.224867:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761312832.224870:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761312832.224879:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1075 00000100:00080000:1.0:1761312832.224882:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761312832.224901:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761312832.224911:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761312832.224925:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a15ef099 x1846869307418176/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761312832.224949:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761312832.224952:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761312832.225046:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307417984:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761312832.225133:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307418112:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761312832.225165:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761312832.225172:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307418048:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761312832.232485:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307418176:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761312832.232518:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307418048:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761312832.232536:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307417984:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761312832.232548:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307418112:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761312832.237219:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761312832.237231:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761312832.237242:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307418176:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761312832.480792:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312832.544874:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312833.504815:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312833.568899:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312834.530817:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312834.592798:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312835.553321:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312835.616925:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312836.576830:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312836.641417:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761312837.345831:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b2298797 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307418240:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761312837.346897:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761312837.346909:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b4eac89a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307418304:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761312837.347942:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761312837.347950:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761312837.347955:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761312837.347970:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1080 00000100:00080000:2.0:1761312837.347974:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761312837.347981:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761312837.347995:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761312837.348014:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000039eebc2 x1846869307418432/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761312837.348042:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761312837.348047:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761312837.348090:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f220504f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307418368:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761312837.351090:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000039eebc2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307418432:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761312837.356269:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b4eac89a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307418304:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761312837.356295:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b2298797 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307418240:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761312837.360033:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761312837.360049:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761312837.360072:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000039eebc2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307418432:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761312837.360901:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f220504f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307418368:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761312837.601200:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312837.665437:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312838.624784:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312838.689024:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312839.649084:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312839.713102:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312840.675603:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312840.737394:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312841.696803:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312841.761344:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761312842.465017:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761312842.465027:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761312842.465031:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761312842.465044:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1085 00000100:00080000:2.0:1761312842.465048:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761312842.465055:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761312842.465066:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761312842.465081:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ba5c8e77 x1846869307418688/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761312842.465105:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761312842.465109:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761312842.465140:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b2298797 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307418560:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761312842.465140:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f220504f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307418496:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761312842.465222:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ba5c8e77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307418688:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761312842.466522:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b4eac89a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307418624:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761312842.486922:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f220504f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307418496:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761312842.490506:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b2298797 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307418560:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761312842.490538:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b4eac89a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307418624:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761312842.490856:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761312842.490862:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761312842.490866:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ba5c8e77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307418688:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761312842.720857:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312842.785507:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312843.744836:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312843.808872:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312844.768978:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312844.832915:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312845.792883:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312845.858159:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312846.817701:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312846.881078:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761312847.584895:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307418752:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761312847.585607:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761312847.585614:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307418816:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761312847.585936:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761312847.585941:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761312847.585944:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761312847.585950:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1090 00000100:00080000:1.0:1761312847.585952:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761312847.585958:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761312847.585966:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761312847.585979:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ed2f9bca x1846869307418944/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1761312847.586041:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307418944:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761312847.586190:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307418880:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761312847.586723:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761312847.586728:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761312847.589421:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307418880:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761312847.589439:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307418816:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761312847.589447:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307418752:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761312847.599746:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761312847.599762:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761312847.599799:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307418944:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761312847.841519:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312847.905494:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312848.864792:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312848.929975:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312849.888793:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312849.952964:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312850.912814:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312850.977289:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312851.936838:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312852.000871:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761312852.705949:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761312852.705960:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761312852.705965:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761312852.705980:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1095 00000100:00080000:2.0:1761312852.705985:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00100000:3.0:1761312852.705987:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b2298797 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307419008:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761312852.705995:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761312852.706031:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761312852.706052:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e9376149 x1846869307419200/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761312852.706081:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761312852.706084:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761312852.706115:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eca34e1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307419136:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761312852.706139:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039978a4e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307419072:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761312852.706146:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e9376149 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307419200:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761312852.707831:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b2298797 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307419008:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761312852.710615:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761312852.710630:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761312852.710679:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e9376149 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307419200:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761312852.711137:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eca34e1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307419136:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761312852.713944:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039978a4e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307419072:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761312852.961430:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312853.025375:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312853.984766:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312854.051382:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312855.008812:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312855.073714:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312856.032881:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312856.098120:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312857.056770:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312857.121981:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761312857.825882:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1761312857.825986:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307419264:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761312857.826083:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007673f690 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307419392:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761312857.826114:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761312857.826124:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000038534f0b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307419328:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761312857.826192:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761312857.826196:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761312857.826201:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1100 00000100:00080000:3.0:1761312857.826204:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761312857.826209:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761312857.826218:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761312857.826234:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000009911ad3 x1846869307419456/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761312857.826259:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761312857.826263:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761312857.826690:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000009911ad3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307419456:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00080000:1.0:1761312857.832385:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761312857.832400:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761312857.832411:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000009911ad3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307419456:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761312857.832523:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000038534f0b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307419328:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761312857.832537:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007673f690 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307419392:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761312857.833385:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307419264:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761312858.080766:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312858.144913:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312859.104777:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312859.168784:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312860.128768:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312860.192843:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312861.153404:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312861.217517:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312862.176810:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312862.241157:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761312862.946179:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761312862.946189:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00100000:0.0:1761312862.946202:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307419520:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761312862.946213:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761312862.946221:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1105 00000100:00080000:1.0:1761312862.946225:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761312862.946233:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761312862.946246:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761312862.946317:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307419648:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761312862.946358:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761312862.946364:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307419584:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761312862.946966:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a1719d14 x1846869307419712/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761312862.946989:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761312862.946994:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761312862.947582:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307419712:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761312862.950618:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307419520:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761312862.951279:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761312862.951286:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761312862.951292:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307419712:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761312862.951588:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307419584:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761312862.951601:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307419648:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761312863.200815:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312863.267097:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312864.225411:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312864.289830:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312865.248813:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312865.312891:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312866.273187:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312866.336883:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312867.297131:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312867.362294:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761312868.064811:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761312868.064818:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761312868.064821:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761312868.064831:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1111 00000100:00080000:1.0:1761312868.064833:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761312868.064838:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761312868.064861:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761312868.064878:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a1719d14 x1846869307419968/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761312868.064904:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761312868.064907:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761312868.064933:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307419840:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761312868.064937:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307419968:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761312868.065028:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1761312868.065036:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307419776:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761312868.065064:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307419904:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761312868.073773:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:1.0:1761312868.073777:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307419776:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761312868.073783:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761312868.073790:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307419968:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761312868.073800:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307419904:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761312868.073814:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307419840:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761312868.320770:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312868.384807:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312869.345583:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312869.408827:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312870.370594:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312870.433007:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312871.392760:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312871.456818:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312872.418202:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312872.480856:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761312873.185152:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761312873.185161:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761312873.185165:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761312873.185177:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1116 00000100:00080000:2.0:1761312873.185181:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761312873.185187:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761312873.185198:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761312873.185215:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ae09c97d x1846869307420224/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761312873.185239:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761312873.185243:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761312873.185271:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aea3302e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307420096:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761312873.185378:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1761312873.185413:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039978a4e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307420032:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761312873.185438:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ffb03e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307420160:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761312873.185591:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ae09c97d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307420224:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761312873.189717:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ffb03e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307420160:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761312873.189738:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aea3302e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307420096:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761312873.191002:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761312873.191009:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761312873.191015:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ae09c97d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307420224:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761312873.191242:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039978a4e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307420032:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761312873.440800:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312873.507949:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312874.464945:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312874.529039:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312875.488786:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312875.553714:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312876.512840:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312876.579484:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312877.536794:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312877.600799:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761312878.305008:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761312878.305017:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761312878.305021:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761312878.305038:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1121 00000100:00080000:1.0:1761312878.305042:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761312878.305051:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761312878.305064:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761312878.305085:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ed2f9bca x1846869307420480/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1761312878.305168:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307420480:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761312878.306056:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307420288:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761312878.306112:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307420416:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761312878.306138:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761312878.306144:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307420352:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761312878.306425:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761312878.306434:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761312878.314992:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307420352:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761312878.315014:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307420288:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761312878.315026:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307420416:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761312878.315191:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761312878.315197:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761312878.315202:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307420480:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761312878.561500:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312878.625931:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312879.586006:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312879.649186:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312880.609044:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312880.672843:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312881.633431:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312881.697436:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312882.656975:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312882.720971:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761312883.425228:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761312883.425235:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761312883.425238:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761312883.425248:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1126 00000100:00080000:1.0:1761312883.425251:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761312883.425257:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761312883.425266:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761312883.425285:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a15ef099 x1846869307420736/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761312883.425309:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761312883.425323:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761312883.425351:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307420608:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761312883.425429:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1761312883.425435:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307420544:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761312883.425457:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307420672:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761312883.428534:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307420736:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761312883.430767:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307420544:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761312883.430785:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307420672:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761312883.430795:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307420608:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761312883.432896:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761312883.432905:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761312883.432913:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307420736:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761312883.680795:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312883.745584:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312884.705174:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312884.768924:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312885.728822:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312885.793498:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312886.753202:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312886.816940:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312887.777340:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312887.841183:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761312888.546364:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761312888.546372:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761312888.546376:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761312888.546388:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1131 00000100:00080000:3.0:1761312888.546392:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761312888.546399:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761312888.546411:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761312888.546426:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f597cda6 x1846869307420992/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761312888.546450:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761312888.546453:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761312888.546483:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307420864:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761312888.546627:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f597cda6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307420992:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761312888.546755:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307420800:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761312888.548180:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000626f98db pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307420928:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761312888.556721:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761312888.556731:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761312888.556755:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f597cda6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307420992:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761312888.556941:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307420800:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761312888.556956:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000626f98db pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307420928:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761312888.557080:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307420864:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761312888.801348:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312888.865187:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312889.828606:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312889.889027:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312890.849696:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312890.913498:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312891.872808:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312891.937997:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312892.896770:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312892.960739:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761312893.665068:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761312893.665075:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761312893.665078:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761312893.665088:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1136 00000100:00080000:3.0:1761312893.665091:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761312893.665097:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000100:00100000:2.0:1761312893.665111:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307421120:192.168.203.151@tcp:400:kworker/u8:0.0 00000008:00100000:3.0:1761312893.665120:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761312893.665137:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000d82b82f x1846869307421248/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761312893.665160:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761312893.665164:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761312893.665197:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307421056:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761312893.665208:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307421248:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761312893.666012:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307421184:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761312893.670461:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307421120:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761312893.670877:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307421184:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761312893.671194:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761312893.671200:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761312893.671204:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307421248:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761312893.671274:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307421056:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761312893.920951:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312893.985004:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312894.945014:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312895.010681:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312895.969710:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312896.032846:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312896.993832:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312897.056871:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312898.016756:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312898.081451:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761312898.784897:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761312898.784905:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761312898.784909:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761312898.784922:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1141 00000100:00080000:3.0:1761312898.784927:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761312898.784934:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761312898.784946:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761312898.784960:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000004505177d x1846869307421504/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761312898.784982:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761312898.784985:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761312898.785036:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307421376:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761312898.785136:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1761312898.785141:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307421312:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761312898.785165:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000101f852a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307421440:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761312898.785778:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307421504:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761312898.790403:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000101f852a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307421440:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761312898.790429:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307421376:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761312898.790488:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307421312:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761312898.790555:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761312898.790559:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761312898.790563:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307421504:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761312899.041260:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312899.105143:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312900.065075:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312900.130008:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312901.089876:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312901.154954:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312902.112802:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312902.176989:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312903.136792:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312903.200824:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761312903.904834:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761312903.904842:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761312903.904846:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761312903.904857:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1146 00000100:00080000:3.0:1761312903.904861:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761312903.904866:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761312903.904875:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761312903.904891:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006c8fcfe5 x1846869307421760/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761312903.904914:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761312903.904917:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761312903.905007:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307421760:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761312903.905132:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307421568:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761312903.905234:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000101f852a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307421696:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761312903.905265:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761312903.905270:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307421632:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761312903.918723:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761312903.918738:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761312903.918751:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307421760:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761312903.932126:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307421632:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761312903.932151:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307421568:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761312903.932162:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000101f852a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307421696:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761312904.160786:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312904.224861:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312905.184781:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312905.248806:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312906.209678:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312906.275541:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312907.232800:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312907.296835:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312908.256766:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312908.320785:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761312909.024987:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761312909.024994:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761312909.024996:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761312909.025006:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1151 00000100:00080000:0.0:1761312909.025008:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761312909.025014:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761312909.025022:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761312909.025035:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000fe4020c4 x1846869307422016/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761312909.025062:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761312909.025065:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761312909.025089:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307421888:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761312909.025175:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1761312909.025196:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307421824:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761312909.025227:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307421952:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761312909.025622:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307422016:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761312909.029792:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307421888:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761312909.034187:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307421824:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761312909.035208:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307421952:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:02020000:3.0:1761312909.035301:0:2262:0:(client.c:1361:ptlrpc_check_status()) 11-0: lustre-OST0000-osc-ffff951bc4d8c800: operation ost_connect to node 192.168.203.151@tcp failed: rc = -19 00000100:00080000:3.0:1761312909.042602:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761312909.042613:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761312909.042624:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307422016:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761312909.280769:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312909.348560:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312910.304833:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312910.369791:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312911.328783:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312911.392807:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312912.352759:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312912.417050:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312913.376761:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312913.440917:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761312914.144876:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761312914.144882:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761312914.144885:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761312914.144895:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1157 00000100:00080000:3.0:1761312914.144899:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761312914.144904:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761312914.144914:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761312914.144930:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000cdcc674d x1846869307422272/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761312914.144959:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761312914.144963:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761312914.145015:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cdcc674d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307422272:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761312914.145067:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307422080:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761312914.145136:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004e00d332 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307422208:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761312914.145164:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761312914.145168:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f133bb2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307422144:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761312914.150542:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761312914.150551:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761312914.150559:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cdcc674d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307422272:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761312914.150700:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f133bb2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307422144:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761312914.150710:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307422080:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761312914.150719:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004e00d332 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307422208:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761312914.400765:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312914.464785:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312915.424773:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312915.490080:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312916.448819:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312916.517187:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312917.472798:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312917.536791:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312918.499820:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312918.563924:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761312919.266021:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761312919.266027:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761312919.266030:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761312919.266040:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1162 00000100:00080000:3.0:1761312919.266053:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761312919.266082:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761312919.266090:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761312919.266103:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f133bb2e x1846869307422528/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761312919.266126:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761312919.266129:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761312919.266370:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f133bb2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307422528:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761312919.267222:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307422336:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761312919.271188:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307422464:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761312919.271272:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761312919.271278:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004e00d332 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307422400:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761312919.277872:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761312919.277881:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761312919.277890:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f133bb2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307422528:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761312919.280585:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307422336:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761312919.286237:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004e00d332 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307422400:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761312919.286251:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307422464:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761312919.528225:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312919.584810:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312920.545198:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312920.608777:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312921.569359:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312921.632869:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312922.592821:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312922.659153:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312923.616785:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312923.680815:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761312924.386883:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761312924.386891:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761312924.386894:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1761312924.386901:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307422592:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761312924.386905:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1167 00000100:00080000:1.0:1761312924.386910:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761312924.386916:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761312924.386926:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761312924.386944:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000540c3ce6 x1846869307422784/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761312924.386974:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761312924.386978:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761312924.386992:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307422720:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761312924.387003:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307422656:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761312924.387287:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307422784:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761312924.390425:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307422656:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761312924.390509:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307422592:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761312924.390521:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307422720:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761312924.390742:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761312924.390747:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761312924.390750:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307422784:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761312924.640755:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312924.706268:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312925.665169:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312925.729710:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312926.693432:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312926.752820:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312927.712771:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312927.778231:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312928.736768:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312928.805878:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761312929.504878:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307422848:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761312929.506179:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761312929.506185:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761312929.506187:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761312929.506194:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1172 00000100:00080000:1.0:1761312929.506196:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761312929.506200:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761312929.506208:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761312929.506219:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000163e3801 x1846869307423040/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761312929.506241:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761312929.506244:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761312929.506297:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307423040:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761312929.506336:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307422912:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761312929.506367:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307422976:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761312929.510439:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307422976:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761312929.510472:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307422848:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761312929.510484:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307422912:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761312929.510623:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761312929.510630:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761312929.510667:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307423040:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761312929.760774:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312929.824924:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312930.784878:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312930.848888:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312931.811166:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312931.874862:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312932.832973:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312932.896803:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312933.856813:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312933.921845:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761312934.624896:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761312934.624905:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761312934.624908:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761312934.624922:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1177 00000100:00080000:3.0:1761312934.624925:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761312934.624933:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761312934.624944:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761312934.624971:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006e93398e x1846869307423296/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761312934.624986:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f133bb2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307423104:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761312934.624996:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761312934.625001:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761312934.625058:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cdcc674d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307423168:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761312934.625066:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307423296:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761312934.625125:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000101f852a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307423232:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761312934.639900:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f133bb2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307423104:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761312934.640159:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cdcc674d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307423168:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761312934.640230:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000101f852a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307423232:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761312934.653762:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761312934.653777:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761312934.653790:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307423296:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761312934.881112:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312934.945558:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312935.904969:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312935.969250:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312936.929461:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312936.993622:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312937.952974:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312938.016970:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312938.976849:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312939.040947:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761312939.745135:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cdcc674d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307423360:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761312939.746552:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761312939.746561:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761312939.746566:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761312939.746577:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1182 00000100:00080000:3.0:1761312939.746583:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761312939.746591:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761312939.746627:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761312939.746670:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000004505177d x1846869307423552/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761312939.746755:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307423552:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761312939.746935:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307423424:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761312939.747200:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761312939.747205:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761312939.747254:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307423488:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761312939.752601:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307423424:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761312939.752745:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761312939.752751:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761312939.752756:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307423552:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761312939.753114:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307423488:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761312939.753127:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cdcc674d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307423360:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761312940.001150:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312940.065374:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312941.025066:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312941.089233:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312942.049165:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312942.112802:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312943.073028:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312943.137494:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312944.096779:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312944.161057:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761312944.864873:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761312944.864888:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761312944.864896:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761312944.864905:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1187 00000100:00080000:1.0:1761312944.864908:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761312944.864914:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761312944.864922:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761312944.864935:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000163e3801 x1846869307423808/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761312944.864953:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761312944.864956:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761312944.864988:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307423808:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761312944.865041:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307423616:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761312944.866481:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307423744:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761312944.866543:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761312944.866550:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307423680:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761312944.869425:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307423680:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761312944.869442:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307423616:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761312944.869452:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307423744:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761312944.869537:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761312944.869542:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761312944.869547:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307423808:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761312945.120763:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312945.184789:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312946.144775:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312946.208880:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312947.168772:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312947.235127:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312948.192811:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312948.256790:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312949.216885:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312949.280855:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761312949.986532:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761312949.986539:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761312949.986542:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761312949.986551:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1192 00000100:00080000:0.0:1761312949.986554:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761312949.986560:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761312949.986570:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761312949.986583:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000bcaee194 x1846869307424064/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761312949.986605:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761312949.986608:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761312949.986661:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307423872:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761312949.986739:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307424000:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761312949.986771:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761312949.986777:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307423936:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761312949.987065:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307424064:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761312949.996809:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307423936:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761312949.996860:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307423872:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761312949.996871:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307424000:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761312949.998117:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761312949.998127:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761312949.998138:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307424064:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761312950.240912:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312950.305012:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312951.264814:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312951.329131:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312952.291110:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312952.353023:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312953.312749:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312953.377706:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312954.337272:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312954.401230:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761312955.105053:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761312955.105063:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761312955.105068:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761312955.105080:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1198 00000100:00080000:3.0:1761312955.105085:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761312955.105092:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761312955.105104:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761312955.105122:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000004c6aebc5 x1846869307424320/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761312955.105197:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307424320:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00080000:3.0:1761312955.105880:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761312955.105890:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761312955.105941:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307424192:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761312955.106027:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1761312955.106034:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cdcc674d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307424128:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761312955.106071:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307424256:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761312955.117582:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761312955.117592:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761312955.117601:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307424320:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761312955.117634:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307424192:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761312955.117704:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cdcc674d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307424128:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761312955.117716:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307424256:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761312955.361843:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312955.424874:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312956.384929:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312956.451084:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312957.408782:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312957.472815:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312958.432757:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312958.496913:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312959.456742:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312959.520886:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761312960.224992:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761312960.225001:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761312960.225005:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761312960.225018:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1203 00000100:00080000:1.0:1761312960.225021:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761312960.225028:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761312960.225059:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761312960.225079:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a15ef099 x1846869307424576/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761312960.225166:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307424576:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761312960.225488:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307424384:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761312960.225524:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307424512:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761312960.225554:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761312960.225560:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307424448:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761312960.225737:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761312960.225742:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761312960.228584:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307424384:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761312960.230043:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307424448:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761312960.230062:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307424512:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761312960.230151:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761312960.230175:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761312960.230180:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307424576:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761312960.480881:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312960.545662:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312961.506946:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312961.568978:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312962.528885:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312962.595039:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312963.552816:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312963.617068:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312964.576831:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312964.642832:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312965.346991:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307424640:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761312965.347297:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761312965.347305:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307424704:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761312965.347377:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761312965.347382:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761312965.347386:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761312965.347392:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1208 00000100:00080000:1.0:1761312965.347395:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761312965.347401:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000100:00100000:1.0:1761312965.347410:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307424768:192.168.203.151@tcp:400:kworker/u8:3.0 00000008:00100000:1.0:1761312965.347483:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761312965.347503:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a15ef099 x1846869307424832/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761312965.347537:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761312965.347558:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761312965.347679:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307424832:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761312965.350558:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307424768:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761312965.350586:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307424704:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761312965.351622:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761312965.351630:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761312965.351681:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307424832:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761312965.351771:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307424640:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761312965.601951:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312965.666189:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312966.624864:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312966.689334:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312967.648833:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312967.712871:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312968.674055:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312968.736893:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312969.696813:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312969.760983:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761312970.465382:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307424896:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761312970.465669:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cdcc674d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307424960:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761312970.465705:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761312970.465711:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761312970.465714:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1761312970.465720:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00080000:3.0:1761312970.465720:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1213 00000100:00080000:3.0:1761312970.465723:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00100000:2.0:1761312970.465730:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307425024:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761312970.465732:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761312970.465746:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761312970.465766:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000626f98db x1846869307425088/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761312970.465780:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761312970.465783:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761312970.465791:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000626f98db pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307425088:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761312970.470606:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307425024:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761312970.470624:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cdcc674d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307424960:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761312970.470776:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307424896:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761312970.471970:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761312970.471977:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761312970.471983:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000626f98db pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307425088:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761312970.720999:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312970.788061:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312971.744942:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312971.810489:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312972.768800:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312972.832829:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312973.796312:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312973.856847:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312974.817854:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312974.880952:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761312975.585152:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761312975.585159:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761312975.585161:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761312975.585171:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1218 00000100:00080000:2.0:1761312975.585173:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761312975.585179:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761312975.585188:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761312975.585201:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008a641e2e x1846869307425344/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761312975.585222:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761312975.585225:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761312975.585308:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008a641e2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307425344:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761312975.585453:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039978a4e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307425152:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761312975.585752:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ffb03e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307425280:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761312975.585794:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761312975.585801:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aea3302e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307425216:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761312975.589163:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aea3302e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307425216:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761312975.589187:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ffb03e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307425280:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761312975.589474:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761312975.589480:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761312975.589485:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008a641e2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307425344:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761312975.589546:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039978a4e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307425152:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761312975.840812:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312975.905404:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312976.865071:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312976.929563:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312977.888787:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312977.953933:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312978.913679:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312978.977795:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312979.936760:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312980.001452:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761312980.704841:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761312980.704848:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761312980.704851:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761312980.704861:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1223 00000100:00080000:3.0:1761312980.704864:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761312980.704870:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761312980.704880:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761312980.704894:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000bfffbafe x1846869307425600/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761312980.704919:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761312980.704922:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761312980.704947:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307425472:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761312980.705040:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1761312980.705046:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307425408:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761312980.705069:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f597cda6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307425536:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761312980.705205:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307425600:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761312980.712293:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307425408:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761312980.712322:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f597cda6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307425536:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761312980.712332:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307425472:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761312980.716041:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761312980.716056:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761312980.716067:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307425600:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761312980.960799:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312981.024875:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312981.986433:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312982.051795:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312983.009009:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312983.073397:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312984.032759:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312984.097432:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312985.057926:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312985.121355:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761312985.825215:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307425664:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761312985.827040:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761312985.827049:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761312985.827053:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761312985.827066:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1228 00000100:00080000:1.0:1761312985.827069:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761312985.827076:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761312985.827088:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761312985.827112:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000163e3801 x1846869307425856/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761312985.827129:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761312985.827132:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761312985.827178:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307425792:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761312985.827274:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761312985.827280:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307425728:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761312985.828458:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307425856:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761312985.829201:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307425664:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761312985.834010:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307425728:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761312985.834036:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307425792:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761312985.834568:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761312985.834575:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761312985.834581:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307425856:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761312986.081564:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312986.144907:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312987.105273:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312987.169685:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312988.128776:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312988.193413:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312989.152764:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312989.217359:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312990.176759:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312990.242038:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761312990.944854:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761312990.944861:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761312990.944864:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761312990.944873:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1233 00000100:00080000:3.0:1761312990.944877:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761312990.944883:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761312990.944893:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761312990.944907:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007673f690 x1846869307426112/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761312990.944931:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761312990.944934:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761312990.944972:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f597cda6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307425984:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761312990.945059:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1761312990.945064:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307425920:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761312990.945088:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307426048:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761312990.945175:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007673f690 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307426112:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761312990.947520:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307425920:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761312990.952167:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307426048:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761312990.952185:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f597cda6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307425984:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761312990.953326:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761312990.953333:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761312990.953339:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007673f690 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307426112:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761312991.200773:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312991.265031:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312992.224908:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312992.288856:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312993.248763:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312993.314382:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312994.272819:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312994.339722:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761312995.297006:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312995.363317:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761312996.065033:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761312996.065042:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761312996.065045:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761312996.065057:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1239 00000100:00080000:0.0:1761312996.065061:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761312996.065067:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761312996.065077:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761312996.065093:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a24c5c6a x1846869307426368/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761312996.065164:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307426368:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00080000:0.0:1761312996.065549:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761312996.065554:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761312996.066285:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307426176:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761312996.066359:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307426304:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761312996.066389:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761312996.066395:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307426240:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761312996.075270:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761312996.075304:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761312996.075317:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307426368:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761312996.075340:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307426240:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761312996.075358:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307426176:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761312996.075371:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307426304:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761312996.321752:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312996.384860:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761312997.344800:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761312997.408793:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761312998.368815:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761312998.433761:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761312999.392796:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761312999.456914:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313000.418215:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761313000.480855:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761313001.185027:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313001.185036:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761313001.185041:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761313001.185053:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1244 00000100:00080000:3.0:1761313001.185058:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761313001.185065:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761313001.185077:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761313001.185095:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000038534f0b x1846869307426624/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761313001.185120:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313001.185123:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761313001.185270:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000038534f0b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307426624:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761313001.186159:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f597cda6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307426432:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313001.186230:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307426560:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313001.186263:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761313001.186270:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307426496:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761313001.189815:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761313001.189829:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761313001.189840:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000038534f0b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307426624:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761313001.189950:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307426496:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313001.189965:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307426560:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313001.191368:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f597cda6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307426432:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313001.440793:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313001.505495:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313002.466984:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313002.529710:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313003.489541:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761313003.552800:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313004.513833:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313004.581999:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313005.536766:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313005.600850:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761313006.304861:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313006.304868:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761313006.304871:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761313006.304880:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1249 00000100:00080000:3.0:1761313006.304884:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761313006.304889:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761313006.304914:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761313006.304935:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e6d9e11e x1846869307426880/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761313006.304959:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313006.304962:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761313006.304983:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000009911ad3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307426688:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313006.305000:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d32bed84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307426752:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313006.305015:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307426880:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761313006.305903:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e58cc722 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307426816:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313006.309248:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000009911ad3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307426688:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313006.309377:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d32bed84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307426752:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761313006.311127:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761313006.311135:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761313006.311143:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307426880:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761313006.311178:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e58cc722 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307426816:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313006.561680:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761313006.624776:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313007.584764:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313007.649038:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313008.610492:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313008.674014:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313009.632755:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761313009.696809:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313010.656794:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313010.721339:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761313011.425075:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313011.425101:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761313011.425105:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761313011.425120:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1254 00000100:00080000:0.0:1761313011.425125:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761313011.425133:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000100:00100000:1.0:1761313011.425145:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307426944:192.168.203.151@tcp:400:kworker/u8:3.0 00000008:00100000:0.0:1761313011.425147:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761313011.425173:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a24c5c6a x1846869307427136/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761313011.425257:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307427136:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761313011.425268:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307427072:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313011.425324:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761313011.425329:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307427008:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761313011.425371:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313011.425376:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761313011.428241:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307426944:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313011.428512:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307427008:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313011.428526:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307427072:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761313011.428588:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761313011.428613:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761313011.428618:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307427136:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761313011.681428:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313011.745752:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313012.704802:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761313012.769663:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313013.728776:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313013.792874:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313014.752772:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313014.817882:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313015.776738:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761313015.841048:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761313016.544877:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313016.544884:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761313016.544887:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761313016.544896:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1259 00000100:00080000:3.0:1761313016.544900:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761313016.544906:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000100:00100000:2.0:1761313016.544909:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d32bed84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307427200:192.168.203.151@tcp:400:kworker/u8:0.0 00000008:00100000:3.0:1761313016.544918:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761313016.544936:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000bc14d8e x1846869307427392/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761313016.545007:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cc6dd8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307427328:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313016.545011:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307427392:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761313016.545050:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761313016.545058:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000041c8b6cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307427264:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761313016.545864:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313016.545870:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761313016.547841:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000041c8b6cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307427264:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313016.547861:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d32bed84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307427200:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313016.547871:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc6dd8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307427328:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761313016.548174:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761313016.548179:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761313016.548183:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307427392:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761313016.800959:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313016.864886:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313017.826516:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313017.890501:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313018.849904:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313018.912874:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313019.873909:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313019.937564:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313020.896761:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313020.961529:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761313021.664871:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313021.664879:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761313021.664882:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761313021.664892:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1264 00000100:00080000:2.0:1761313021.664896:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761313021.664901:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761313021.664912:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761313021.664927:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000015f165e6 x1846869307427648/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761313021.664954:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313021.664984:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761313021.665012:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002292ab57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307427520:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313021.665100:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1761313021.665107:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e3466f5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307427456:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313021.665133:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b3aaddc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307427584:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313021.665527:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000015f165e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307427648:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761313021.670626:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e3466f5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307427456:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313021.670683:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b3aaddc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307427584:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313021.670694:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002292ab57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307427520:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761313021.672351:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761313021.672359:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761313021.672366:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000015f165e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307427648:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761313021.921256:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313021.984859:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313022.944768:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313023.010082:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313023.971563:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313024.035217:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313024.992873:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313025.056901:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313026.016814:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313026.081004:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761313026.784900:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313026.784910:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761313026.784913:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761313026.784926:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1269 00000100:00080000:2.0:1761313026.784930:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761313026.784937:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761313026.784949:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761313026.784967:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e662bceb x1846869307427904/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761313026.784994:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313026.784997:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761313026.785077:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002292ab57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307427712:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313026.785099:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b3aaddc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307427776:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313026.785189:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e3466f5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307427840:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313026.786447:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e662bceb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307427904:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761313026.791712:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002292ab57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307427712:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313026.791741:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e3466f5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307427840:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313026.791897:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b3aaddc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307427776:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761313026.794020:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761313026.794034:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761313026.794046:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e662bceb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307427904:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761313027.040800:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313027.104866:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313028.066029:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313028.129257:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313029.089457:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313029.153894:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313030.113265:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313030.176864:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313031.136765:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313031.200865:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761313031.904792:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313031.904800:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761313031.904803:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761313031.904814:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1274 00000100:00080000:2.0:1761313031.904817:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761313031.904823:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761313031.904833:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761313031.904847:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000e1be9a7 x1846869307428160/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761313031.904871:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313031.904874:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761313031.904964:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e1be9a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307428160:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761313031.905119:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ee08055d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307428032:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313031.905153:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1761313031.905158:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b3aaddc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307427968:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313031.905179:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000011b5c99f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307428096:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761313031.909332:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761313031.909344:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761313031.909353:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e1be9a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307428160:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761313031.909804:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ee08055d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307428032:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313031.911195:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b3aaddc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307427968:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313031.914074:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000011b5c99f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307428096:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313032.161027:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313032.224908:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313033.184800:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313033.251844:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313034.208912:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313034.272866:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313035.233075:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313035.300472:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313036.256760:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313036.324331:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761313037.026438:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313037.026446:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761313037.026449:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761313037.026459:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1279 00000100:00080000:0.0:1761313037.026462:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761313037.026467:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761313037.026477:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761313037.026507:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008e59d025 x1846869307428416/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761313037.026532:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313037.026535:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761313037.026572:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307428224:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761313037.026590:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307428416:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761313037.027257:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307428352:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761313037.027292:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761313037.027297:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307428288:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761313037.033872:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307428224:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761313037.033895:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307428352:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761313037.033994:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761313037.034000:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761313037.034004:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307428416:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761313037.034909:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307428288:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761313037.280759:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313037.346505:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313038.304946:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313038.368912:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313039.331928:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313039.394039:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313040.352839:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313040.416875:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313041.376778:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313041.440847:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761313042.146062:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313042.146069:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761313042.146073:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761313042.146083:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1285 00000100:00080000:3.0:1761313042.146085:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761313042.146091:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761313042.146101:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761313042.146114:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d32bed84 x1846869307428672/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761313042.146137:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313042.146139:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761313042.146162:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307428544:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761313042.146235:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d32bed84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307428672:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761313042.146390:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f364ec74 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307428480:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761313042.148485:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cc6dd8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307428608:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761313042.154316:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f364ec74 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307428480:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761313042.154346:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc6dd8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307428608:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761313042.154364:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307428544:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761313042.154680:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761313042.154686:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761313042.154691:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d32bed84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307428672:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761313042.402492:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313042.474259:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313043.426412:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313043.492149:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313044.448899:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313044.514489:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313045.476665:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313045.536851:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313046.496768:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313046.562078:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313047.265968:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307428736:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761313047.266385:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761313047.266394:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307428800:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761313047.266666:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313047.266671:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761313047.266674:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761313047.266679:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1290 00000100:00080000:0.0:1761313047.266682:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761313047.266687:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761313047.266695:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761313047.266711:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e00b5e9d x1846869307428928/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761313047.266786:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307428928:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761313047.267630:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761313047.267690:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307428864:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761313047.268613:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313047.268619:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761313047.277088:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307428864:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761313047.277785:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307428800:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761313047.277800:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307428736:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761313047.277897:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313047.277903:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313047.277907:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307428928:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761313047.520974:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313047.584815:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313048.544786:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313048.611251:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313049.568765:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313049.633718:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313050.593802:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313050.657158:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313051.617290:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313051.681198:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761313052.384858:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313052.384875:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761313052.384877:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761313052.384886:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1295 00000100:00080000:1.0:1761313052.384889:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761313052.384894:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761313052.384903:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761313052.384918:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ed2f9bca x1846869307429184/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761313052.384939:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313052.384958:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313052.385034:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307428992:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313052.385144:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307429120:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313052.385179:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761313052.385185:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307429056:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313052.385735:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307429184:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00080000:3.0:1761313052.398949:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313052.398963:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313052.398972:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307429184:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761313052.398982:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307429120:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313052.399013:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307429056:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313052.399024:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307428992:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313052.642576:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313052.704805:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313053.664761:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313053.728840:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313054.688751:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313054.753614:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313055.714957:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313055.776842:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313056.736752:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313056.801391:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761313057.505279:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313057.505286:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761313057.505288:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761313057.505303:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1300 00000100:00080000:0.0:1761313057.505306:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761313057.505311:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761313057.505321:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761313057.505337:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000034a6cd79 x1846869307429440/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1761313057.505415:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307429440:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761313057.505840:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307429248:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761313057.505875:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307429376:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761313057.505905:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761313057.505911:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307429312:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761313057.506163:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313057.506169:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00080000:3.0:1761313057.514108:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313057.514118:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313057.514127:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307429440:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761313057.524453:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307429376:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761313057.524784:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307429312:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761313057.524812:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307429248:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761313057.760812:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313057.825952:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313058.785235:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313058.848835:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313059.808811:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313059.874230:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313060.832766:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313060.897095:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313061.857831:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313061.921246:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761313062.630272:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313062.630279:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761313062.630282:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761313062.630291:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1305 00000100:00080000:3.0:1761313062.630294:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761313062.630300:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761313062.630308:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761313062.630323:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e6d9e11e x1846869307429696/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761313062.630352:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313062.630356:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313062.630418:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307429696:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761313062.630584:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307429504:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761313062.630620:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e58cc722 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307429632:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761313062.630686:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761313062.630693:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000041c8b6cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307429568:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761313062.634464:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000041c8b6cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307429568:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761313062.634482:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307429504:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761313062.634491:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e58cc722 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307429632:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:02020000:0.0:1761313062.634586:0:2262:0:(client.c:1361:ptlrpc_check_status()) 11-0: lustre-OST0000-osc-ffff951bc4d8c800: operation ost_connect to node 192.168.203.151@tcp failed: rc = -19 00000100:00080000:0.0:1761313062.644075:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761313062.644086:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761313062.644099:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307429696:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761313062.880762:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313062.947606:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313063.904798:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313063.968843:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313064.929761:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313064.992847:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313065.953978:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313066.019724:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313066.976816:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313067.044855:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761313067.745217:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307429760:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761313067.745470:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313067.745477:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761313067.745480:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761313067.745486:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1310 00000100:00080000:1.0:1761313067.745490:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761313067.745496:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761313067.745505:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761313067.745519:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000003871442b x1846869307429952/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1761313067.745591:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307429952:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761313067.746390:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307429888:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313067.746409:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761313067.746414:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307429824:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761313067.747711:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313067.747717:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00080000:3.0:1761313067.758225:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313067.758240:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313067.758252:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307429952:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761313067.761128:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307429824:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313067.761146:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307429888:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313067.761155:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307429760:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313068.003070:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313068.070523:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313069.024765:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313069.093960:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313070.048750:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313070.112863:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313071.072769:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313071.136842:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313072.096793:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313072.161013:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761313072.868205:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313072.868232:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761313072.868236:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761313072.868248:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1315 00000100:00080000:1.0:1761313072.868253:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761313072.868261:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761313072.868273:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761313072.868296:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000003871442b x1846869307430208/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761313072.868321:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313072.868324:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761313072.868369:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307430080:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313072.868412:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307430208:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761313072.868474:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1761313072.868485:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307430016:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313072.868521:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307430144:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313072.877747:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307430016:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313072.877774:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307430144:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313072.877786:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307430080:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761313072.882789:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313072.882803:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313072.882815:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307430208:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761313073.121043:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313073.186536:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313074.145784:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313074.209767:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313075.168773:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313075.232997:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313076.192773:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313076.257373:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313077.216866:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313077.281598:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761313077.984913:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313077.984920:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761313077.984923:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761313077.984941:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1320 00000100:00080000:1.0:1761313077.984957:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761313077.984962:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761313077.984971:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761313077.984987:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a1719d14 x1846869307430464/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761313077.985010:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313077.985013:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761313077.985083:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307430464:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761313077.986476:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307430272:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313077.987017:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307430400:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313077.987054:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761313077.987108:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307430336:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761313077.991751:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313077.991781:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313077.991794:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307430464:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761313077.991907:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307430336:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313077.991926:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307430272:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313077.991947:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307430400:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313078.240835:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313078.306513:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313079.264793:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313079.330043:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313080.290576:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313080.353097:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313081.312789:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313081.376859:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313082.337525:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313082.402230:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761313083.109975:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313083.109995:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761313083.109999:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761313083.110012:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1326 00000100:00080000:0.0:1761313083.110017:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761313083.110024:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761313083.110036:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761313083.110057:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b5a39135 x1846869307430720/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761313083.110081:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313083.110085:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313083.110121:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307430592:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761313083.110235:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1761313083.110241:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307430528:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761313083.110268:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307430656:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761313083.110520:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307430720:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761313083.117099:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307430528:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761313083.117123:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307430656:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761313083.117133:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307430592:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761313083.117224:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313083.117229:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313083.117234:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307430720:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761313083.360857:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313083.425503:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313084.384823:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313084.450655:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313085.408831:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313085.473569:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313086.432794:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313086.498030:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313087.457469:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313087.521563:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313088.224972:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000009911ad3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307430784:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761313088.225540:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761313088.225548:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f597cda6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307430848:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761313088.226546:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307430912:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761313088.226748:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313088.226753:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761313088.226756:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761313088.226761:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1331 00000100:00080000:3.0:1761313088.226763:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761313088.226781:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761313088.226791:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761313088.226806:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000d82b82f x1846869307430976/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761313088.226856:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307430976:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00080000:3.0:1761313088.228287:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313088.228295:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00080000:3.0:1761313088.235537:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313088.235545:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313088.235553:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307430976:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761313088.237473:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000009911ad3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307430784:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761313088.237508:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307430912:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761313088.237517:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f597cda6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307430848:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761313088.481275:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313088.545089:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313089.506080:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313089.569325:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313090.528788:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313090.592832:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313091.552758:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313091.616997:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313092.579325:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313092.640897:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761313093.344875:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313093.344883:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761313093.344886:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761313093.344897:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1336 00000100:00080000:3.0:1761313093.344900:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761313093.344906:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761313093.344915:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761313093.344944:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000bfffbafe x1846869307431232/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761313093.344975:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313093.344979:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313093.345081:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307431232:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761313093.346945:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307431040:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761313093.347027:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007673f690 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307431168:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761313093.347083:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761313093.347090:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000038534f0b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307431104:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761313093.352615:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761313093.352625:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761313093.352634:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307431232:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761313093.352741:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007673f690 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307431168:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761313093.352760:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307431040:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761313093.356275:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000038534f0b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307431104:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761313093.600801:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313093.666203:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313094.624778:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313094.690525:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313095.648825:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313095.715060:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313096.672840:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313096.737661:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313097.700073:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313097.760953:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313098.464873:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000626f98db pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307431296:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761313098.466811:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313098.466817:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761313098.466819:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761313098.466828:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1341 00000100:00080000:3.0:1761313098.466832:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761313098.466836:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761313098.466845:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761313098.466859:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000004c6aebc5 x1846869307431488/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761313098.466901:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307431488:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761313098.467712:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cdcc674d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307431360:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761313098.467771:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761313098.467777:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307431424:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761313098.467968:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313098.467973:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761313098.469605:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000626f98db pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307431296:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761313098.471508:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761313098.471518:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761313098.471526:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307431488:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761313098.471625:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307431424:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761313098.471663:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cdcc674d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307431360:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761313098.721389:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313098.784858:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313099.745619:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313099.810340:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313100.770518:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313100.833912:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313101.792906:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313101.857151:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313102.816808:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313102.880862:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761313103.590946:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313103.590956:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761313103.590960:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761313103.590974:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1346 00000100:00080000:3.0:1761313103.590978:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761313103.590985:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761313103.590998:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761313103.591016:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000101f852a x1846869307431744/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761313103.591039:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313103.591043:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313103.591182:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000101f852a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307431744:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761313103.591280:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307431552:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761313103.591355:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307431680:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761313103.591389:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761313103.591396:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307431616:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761313103.598239:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761313103.598253:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761313103.598266:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000101f852a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307431744:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761313103.603285:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307431552:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761313103.605006:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307431680:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761313103.606890:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307431616:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761313103.840965:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313103.905455:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313104.864792:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313104.931192:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313105.888774:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313105.953513:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313106.912998:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313106.978657:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313107.936780:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313108.000865:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761313108.704867:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313108.704876:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761313108.704894:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761313108.704903:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1351 00000100:00080000:3.0:1761313108.704906:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761313108.704912:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761313108.704922:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761313108.704939:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000602b09d0 x1846869307432000/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761313108.704959:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313108.704962:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761313108.705055:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f133bb2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307431808:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761313108.705150:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004e00d332 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307431936:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761313108.705156:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307432000:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761313108.705189:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761313108.705198:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307431872:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761313108.708807:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004e00d332 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307431936:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761313108.710216:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307431872:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761313108.710233:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f133bb2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307431808:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761313108.710355:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761313108.710361:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761313108.710377:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307432000:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761313108.962008:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313109.024975:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313109.984950:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313110.050609:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313111.009422:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313111.073448:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313112.034624:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313112.099305:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313113.058539:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313113.121149:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313113.825398:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307432064:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761313113.826372:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313113.826379:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761313113.826382:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761313113.826390:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1356 00000100:00080000:0.0:1761313113.826393:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761313113.826400:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761313113.826409:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761313113.826424:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000094820351 x1846869307432256/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761313113.826448:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313113.826451:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313113.826623:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307432128:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761313113.826705:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761313113.826712:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307432192:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761313113.826932:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307432256:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761313113.831896:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307432064:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761313113.833877:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313113.833885:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313113.833893:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307432256:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761313113.834062:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307432192:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761313113.834072:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307432128:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761313114.080776:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313114.145070:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313115.104769:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313115.168876:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313116.128843:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313116.195203:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313117.152806:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313117.216906:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313118.176778:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313118.240872:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761313118.944834:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313118.944842:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761313118.944845:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761313118.944855:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1361 00000100:00080000:0.0:1761313118.944858:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761313118.944878:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761313118.944887:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761313118.944903:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e00b5e9d x1846869307432512/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761313118.944931:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313118.944934:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761313118.945019:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307432320:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761313118.945099:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307432448:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761313118.945130:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761313118.945136:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307432384:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761313118.947516:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307432512:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761313118.949182:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307432384:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761313118.949216:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307432320:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761313118.949227:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307432448:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761313118.950150:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313118.950163:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313118.950174:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307432512:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761313119.200794:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313119.265932:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313120.224759:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313120.289812:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313121.248801:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313121.314203:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313122.272726:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313122.336867:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313123.296880:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313123.361343:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761313124.065869:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313124.065877:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761313124.065880:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761313124.065891:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1367 00000100:00100000:0.0:1761313124.065892:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307432576:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761313124.065895:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761313124.065913:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761313124.065922:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761313124.065937:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000eb8b1f19 x1846869307432768/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761313124.065959:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313124.065962:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761313124.065983:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307432640:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313124.065986:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307432704:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313124.066014:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307432768:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00080000:3.0:1761313124.069318:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313124.069328:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313124.069337:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307432768:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761313124.069493:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307432640:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313124.069505:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307432576:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313124.069522:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307432704:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313124.320973:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313124.384886:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313125.344820:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313125.410062:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313126.369098:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313126.434935:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313127.395855:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313127.457429:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313128.416759:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313128.480857:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761313129.187015:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313129.187022:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761313129.187025:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761313129.187033:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1372 00000100:00080000:3.0:1761313129.187037:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761313129.187042:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761313129.187052:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761313129.187064:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000004e00d332 x1846869307433024/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761313129.187087:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313129.187090:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313129.187168:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004e00d332 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307433024:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761313129.187421:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307432832:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761313129.187472:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307432960:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761313129.187494:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761313129.187499:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f133bb2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307432896:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761313129.191103:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313129.191116:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313129.191128:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004e00d332 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307433024:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761313129.193170:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307432832:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761313129.197189:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f133bb2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307432896:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761313129.197216:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307432960:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313129.441581:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313129.504876:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313130.466233:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313130.530485:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313131.488808:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313131.554498:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313132.512861:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313132.577673:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313133.537020:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313133.600879:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761313134.304895:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313134.304903:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761313134.304906:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761313134.304916:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1377 00000100:00080000:1.0:1761313134.304919:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761313134.304925:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761313134.304951:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761313134.304968:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a1719d14 x1846869307433280/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761313134.304992:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313134.305005:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313134.305089:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307433088:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313134.305175:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307433216:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313134.305208:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761313134.305213:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307433152:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313134.305457:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307433280:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761313134.313619:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307433152:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313134.313667:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307433088:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313134.313680:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307433216:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761313134.337950:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313134.337960:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313134.337969:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307433280:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761313134.560765:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313134.624886:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313135.587989:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313135.649073:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313136.608784:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313136.679724:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313137.633007:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313137.696875:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313138.656771:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313138.720840:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761313139.424877:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313139.424884:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761313139.424888:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761313139.424898:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1382 00000100:00080000:3.0:1761313139.424902:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761313139.424908:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000100:00100000:2.0:1761313139.424913:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004e00d332 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307433344:192.168.203.151@tcp:400:kworker/u8:2.0 00000008:00100000:3.0:1761313139.424917:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761313139.424933:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000004505177d x1846869307433536/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761313139.424959:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313139.424962:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761313139.424999:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307433408:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761313139.425023:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307433472:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761313139.425107:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307433536:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761313139.428803:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004e00d332 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307433344:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761313139.433939:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307433472:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761313139.434396:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307433408:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761313139.435438:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761313139.435444:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761313139.435450:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307433536:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761313139.681855:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313139.744864:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313140.704801:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313140.769314:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313141.728803:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313141.794169:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313142.755319:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313142.817880:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313143.779674:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313143.843165:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761313144.544965:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313144.544978:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761313144.544984:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1761313144.544995:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307433600:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761313144.544996:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1387 00000100:00080000:1.0:1761313144.545002:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761313144.545012:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761313144.545026:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761313144.545047:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000163e3801 x1846869307433792/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761313144.545076:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313144.545079:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761313144.545129:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307433664:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313144.545186:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307433792:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761313144.545197:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307433728:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313144.549802:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307433600:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761313144.552211:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313144.552225:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313144.552233:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307433792:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761313144.552312:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307433664:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313144.552357:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307433728:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313144.801780:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313144.868426:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313145.824814:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313145.890274:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313146.849060:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313146.915865:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313147.873082:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313147.938473:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313148.897753:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313148.962573:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761313149.664937:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307433856:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313149.665477:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307433920:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761313149.665489:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313149.665495:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761313149.665499:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761313149.665504:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1392 00000100:00080000:1.0:1761313149.665508:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00100000:0.0:1761313149.665509:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761313149.665513:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307433984:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761313149.665534:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761313149.665548:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761313149.665568:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000eb8b1f19 x1846869307434048/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1761313149.665681:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307434048:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00080000:1.0:1761313149.668965:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313149.668979:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313149.671820:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307433984:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313149.671844:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307433920:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313149.671904:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307433856:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761313149.673481:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313149.673493:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313149.673502:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307434048:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761313149.920810:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313149.987064:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313150.945006:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313151.009743:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313151.968831:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313152.034549:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313152.994343:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313153.058503:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313154.016735:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313154.081505:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761313154.784966:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313154.784976:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761313154.784982:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761313154.785019:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1397 00000100:00080000:3.0:1761313154.785024:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761313154.785033:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000100:00100000:2.0:1761313154.785043:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307434112:192.168.203.151@tcp:400:kworker/u8:0.0 00000008:00100000:3.0:1761313154.785049:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761313154.785163:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cdcc674d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307434240:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313154.785196:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761313154.785203:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000101f852a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307434176:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313154.785557:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006c8fcfe5 x1846869307434304/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761313154.785668:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307434304:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00080000:3.0:1761313154.787442:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313154.787454:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761313154.794547:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307434112:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313154.794569:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cdcc674d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307434240:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761313154.797426:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761313154.797436:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761313154.797444:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307434304:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761313154.799228:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000101f852a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307434176:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313155.040768:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313155.104864:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313156.065162:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313156.129893:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313157.088760:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313157.157077:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313158.112805:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313158.176913:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313159.136962:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313159.200847:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761313159.905733:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313159.905742:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761313159.905746:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761313159.905760:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1402 00000100:00080000:1.0:1761313159.905765:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761313159.905772:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761313159.905784:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761313159.905801:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000009cec3faa x1846869307434560/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761313159.905828:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313159.905831:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313159.906279:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307434368:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313159.906422:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307434496:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313159.906462:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761313159.906492:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307434432:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313159.906603:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307434560:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761313159.921885:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307434496:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313159.921938:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307434432:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761313159.922776:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313159.922783:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313159.922790:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307434560:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761313159.923190:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307434368:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313160.163673:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313160.224866:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313161.186103:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313161.251248:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761313161.953596:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307434624:192.168.203.151@tcp:17:kworker/1:0.0 00000100:00100000:0.0:1761313161.953714:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761313161.953721:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307434688:192.168.203.151@tcp:17:kworker/1:0.0 00000100:00100000:0.0:1761313161.960842:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307434624:192.168.203.151@tcp:17:kworker/1:0.0 00000100:00100000:0.0:1761313161.962082:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307434688:192.168.203.151@tcp:17:kworker/1:0.0 00000100:00100000:1.0:1761313162.210675:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313162.274403:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313163.234282:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313163.296840:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313164.256759:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313164.320952:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761313165.025029:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313165.025038:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761313165.025041:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761313165.025052:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1407 00000100:00080000:3.0:1761313165.025056:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761313165.025078:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761313165.025088:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761313165.025104:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000d82b82f x1846869307434944/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761313165.025183:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307434944:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761313165.025235:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307434752:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313165.025292:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000038534f0b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307434880:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313165.025343:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761313165.025350:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000626f98db pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307434816:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761313165.025912:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313165.025916:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761313165.029919:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000626f98db pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307434816:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313165.029936:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307434752:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313165.029946:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000038534f0b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307434880:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761313165.030085:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761313165.030090:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761313165.030094:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307434944:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761313165.280755:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313165.344848:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313166.304769:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313166.370179:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313167.328792:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313167.393219:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313168.352767:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313168.419150:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313169.376767:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313169.441209:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761313170.144837:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313170.144846:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761313170.144850:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761313170.144859:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1413 00000100:00080000:3.0:1761313170.144863:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761313170.144868:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761313170.144877:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761313170.144892:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a183f7ef x1846869307435200/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761313170.144915:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313170.144918:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313170.145006:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307435200:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761313170.145453:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007673f690 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307435008:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313170.145487:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f597cda6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307435136:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313170.145510:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761313170.145515:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307435072:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313170.154579:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307435072:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313170.154597:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007673f690 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307435008:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313170.154607:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f597cda6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307435136:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761313170.158632:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761313170.158655:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761313170.158664:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307435200:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761313170.400779:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313170.465227:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313171.424802:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313171.493462:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313172.449798:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313172.515162:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313173.473284:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313173.540811:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313174.496724:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313174.561351:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761313175.264887:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313175.264895:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761313175.264898:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761313175.264907:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1418 00000100:00080000:1.0:1761313175.264911:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761313175.264916:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761313175.264925:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761313175.264937:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a15ef099 x1846869307435456/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761313175.264957:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313175.264961:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761313175.264984:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307435328:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313175.265066:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1761313175.265072:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307435264:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313175.265096:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307435392:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313175.265327:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307435456:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761313175.269693:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307435264:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313175.275304:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307435392:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313175.275322:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307435328:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761313175.275672:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313175.275677:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313175.275695:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307435456:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761313175.520767:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313175.584839:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313176.547492:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313176.608795:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313177.568790:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313177.634007:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313178.592792:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313178.656848:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313179.616774:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313179.683224:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313180.385241:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000009911ad3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307435520:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761313180.390177:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313180.390186:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761313180.390190:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761313180.390203:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1423 00000100:00080000:3.0:1761313180.390217:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00100000:2.0:1761313180.391462:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307435584:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313180.391599:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761313180.391607:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e58cc722 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307435648:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313180.392028:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000009911ad3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307435520:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761313180.392327:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761313180.392343:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761313180.392375:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000bc14d8e x1846869307435712/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761313180.392409:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313180.392413:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313180.392433:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307435712:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761313180.402186:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e58cc722 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307435648:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313180.402278:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307435584:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761313180.402438:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313180.402444:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313180.402449:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307435712:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761313180.640824:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313180.705611:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313181.664922:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313181.728998:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313182.688786:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313182.753113:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313183.712774:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313183.776915:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313184.737996:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313184.800847:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761313185.504905:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313185.504915:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761313185.504920:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761313185.504934:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1428 00000100:00080000:3.0:1761313185.504939:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00100000:2.0:1761313185.504984:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307435776:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313185.505134:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d32bed84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307435904:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313185.505190:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761313185.505218:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000041c8b6cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307435840:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761313185.509297:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761313185.509316:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761313185.509349:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000cc6dd8ca x1846869307435968/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761313185.509383:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313185.509387:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313185.512725:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cc6dd8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307435968:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761313185.513145:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307435776:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313185.513161:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d32bed84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307435904:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313185.515918:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000041c8b6cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307435840:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761313185.519581:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761313185.519598:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761313185.519610:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc6dd8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307435968:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761313185.761834:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313185.824845:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313186.785416:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313186.848872:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313187.809520:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313187.872854:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313188.832779:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313188.898187:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313189.860497:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313189.921166:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761313190.624838:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313190.624846:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761313190.624850:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1761313190.624857:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000011b5c99f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307436032:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761313190.624859:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1433 00000100:00080000:2.0:1761313190.624864:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761313190.624871:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761313190.624882:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761313190.624899:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000e1be9a7 x1846869307436224/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1761313190.624957:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ee08055d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307436160:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313190.624960:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e1be9a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307436224:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761313190.624997:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761313190.625004:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b3aaddc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307436096:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761313190.625021:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313190.625026:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761313190.629465:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000011b5c99f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307436032:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313190.629487:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ee08055d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307436160:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761313190.630833:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761313190.630841:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761313190.630848:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e1be9a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307436224:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761313190.631168:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b3aaddc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307436096:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313190.880828:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313190.946544:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313191.907281:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313191.969628:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313192.932240:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313192.993146:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313193.952802:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313194.016880:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313194.982307:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313195.051302:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313195.776632:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f364ec74 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307436288:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313195.777742:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cc6dd8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307436352:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761313195.788701:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313195.788708:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761313195.788711:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761313195.788721:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1438 00000100:00080000:3.0:1761313195.788724:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761313195.788729:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761313195.788755:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761313195.789078:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d32bed84 x1846869307436480/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761313195.789106:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313195.789109:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761313195.789137:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000041c8b6cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307436416:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313195.789225:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f364ec74 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307436288:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313195.789266:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc6dd8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307436352:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313195.789342:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d32bed84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307436480:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00080000:0.0:1761313195.798177:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761313195.798187:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761313195.798196:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d32bed84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307436480:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761313195.798273:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000041c8b6cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307436416:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313196.000886:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313196.068542:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313197.024765:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313197.091537:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313198.049654:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313198.113369:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313199.073315:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313199.136872:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313200.097549:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313200.160843:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761313200.864870:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313200.864878:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761313200.864882:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761313200.864894:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1443 00000100:00080000:3.0:1761313200.864898:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761313200.864904:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761313200.864916:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761313200.864934:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000bc14d8e x1846869307436736/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761313200.864953:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000041c8b6cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307436544:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761313200.864967:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313200.864972:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761313200.865014:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cc6dd8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307436608:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313200.865147:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f364ec74 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307436672:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313200.865260:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307436736:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761313200.870489:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc6dd8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307436608:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313200.870601:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000041c8b6cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307436544:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761313200.870628:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761313200.870634:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761313200.870668:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307436736:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761313200.870721:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f364ec74 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307436672:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313201.120780:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313201.186764:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313202.144812:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313202.210191:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313203.171296:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313203.236021:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313204.192809:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313204.256851:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313205.217695:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313205.282368:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761313205.986232:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313205.986241:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761313205.986245:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761313205.986256:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1448 00000100:00080000:3.0:1761313205.986259:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761313205.986265:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761313205.986275:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761313205.986290:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000009911ad3 x1846869307436992/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761313205.986315:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313205.986318:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313205.986408:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000009911ad3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307436992:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761313205.986763:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cc6dd8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307436800:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313205.986812:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e58cc722 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307436928:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313205.986840:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761313205.986846:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307436864:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313205.996617:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307436864:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313205.996671:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e58cc722 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307436928:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761313205.999786:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761313205.999796:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761313205.999805:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000009911ad3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307436992:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761313205.999897:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc6dd8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307436800:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313206.240766:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313206.305134:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313207.268438:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313207.328866:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313208.288741:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313208.353099:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313209.313900:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313209.378969:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313210.336802:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313210.408096:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761313211.104958:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313211.104997:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761313211.105002:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761313211.105016:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1454 00000100:00080000:1.0:1761313211.105021:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00100000:0.0:1761313211.105029:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307437056:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761313211.105033:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761313211.105056:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761313211.105080:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ed2f9bca x1846869307437248/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761313211.105108:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313211.105113:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761313211.105145:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307437120:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313211.105152:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307437184:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313211.105311:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307437248:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761313211.114405:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307437120:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761313211.114530:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313211.114536:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313211.114541:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307437248:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761313211.118278:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307437056:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313211.118323:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307437184:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313211.360774:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313211.424901:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313212.385161:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313212.449041:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313213.408764:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313213.472924:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313214.433533:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313214.496874:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313215.456763:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313215.520866:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761313216.227558:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313216.227567:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761313216.227570:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761313216.227582:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1459 00000100:00080000:0.0:1761313216.227585:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761313216.227592:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761313216.227603:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761313216.227619:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007dcceb4a x1846869307437504/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761313216.227701:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313216.227704:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313216.227732:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307437376:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313216.227807:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307437504:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761313216.227831:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1761313216.227845:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307437312:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313216.227898:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307437440:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313216.233493:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307437312:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313216.233514:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307437440:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313216.233524:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307437376:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:02020000:3.0:1761313216.233674:0:2262:0:(client.c:1361:ptlrpc_check_status()) 11-0: lustre-OST0000-osc-ffff951bc4d8c800: operation ost_connect to node 192.168.203.151@tcp failed: rc = -19 00000100:00080000:3.0:1761313216.248119:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313216.248129:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313216.248141:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307437504:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761313216.481040:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313216.545040:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313217.504758:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313217.568869:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313218.528765:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313218.593419:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313219.556597:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313219.616853:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313220.577431:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313220.641564:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761313221.344885:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313221.344891:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761313221.344895:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761313221.344905:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1464 00000100:00080000:3.0:1761313221.344909:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761313221.344925:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761313221.344934:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761313221.344949:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000bfffbafe x1846869307437760/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761313221.344971:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313221.344974:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761313221.345056:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307437568:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313221.345146:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007673f690 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307437696:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313221.345180:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761313221.345186:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f597cda6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307437632:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313221.345274:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307437760:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761313221.352769:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007673f690 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307437696:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313221.359105:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f597cda6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307437632:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313221.361965:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307437568:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761313221.362067:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313221.362077:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313221.362085:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307437760:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761313221.601093:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313221.664846:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313222.624806:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313222.691255:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313223.648794:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313223.712869:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313224.672797:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313224.738163:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313225.696823:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313225.761238:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761313226.464916:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313226.464926:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761313226.464930:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761313226.464944:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1469 00000100:00080000:3.0:1761313226.464948:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761313226.464954:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761313226.464967:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761313226.464984:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000004c6aebc5 x1846869307438016/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761313226.465018:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313226.465022:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313226.465242:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307438016:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761313226.466266:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307437824:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313226.466315:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000038534f0b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307437952:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313226.466381:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761313226.466389:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307437888:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761313226.472018:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313226.472033:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313226.472046:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307438016:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761313226.472161:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307437888:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313226.472172:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307437824:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313226.472182:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000038534f0b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307437952:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313226.721200:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313226.786270:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313227.744847:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313227.808930:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313228.768782:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313228.834287:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313229.793722:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313229.858936:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313230.817005:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313230.883002:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313231.588452:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307438080:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313231.590592:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000626f98db pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307438144:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313231.591725:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761313231.591738:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000101f852a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307438208:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313231.591853:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307438080:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761313231.591942:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313231.591969:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761313231.591973:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761313231.591980:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1474 00000100:00080000:3.0:1761313231.591984:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761313231.591990:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761313231.592005:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761313231.592030:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006c8fcfe5 x1846869307438272/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761313231.592060:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313231.592065:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313231.594955:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307438272:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761313231.595099:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000626f98db pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307438144:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313231.595347:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000101f852a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307438208:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761313231.596943:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313231.596955:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313231.596964:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307438272:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761313231.840758:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313231.904893:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313232.864794:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313232.929561:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313233.888797:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313233.952904:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313234.913588:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313234.978295:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313235.937522:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313236.002523:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761313236.705952:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313236.705960:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761313236.705964:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761313236.705974:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1479 00000100:00080000:1.0:1761313236.705978:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 6s 00000100:00080000:1.0:1761313236.705984:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761313236.705994:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761313236.706023:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ed2f9bca x1846869307438528/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761313236.706041:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313236.706044:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761313236.706074:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307438400:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313236.706165:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1761313236.706173:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307438336:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313236.706199:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307438464:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313236.706906:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307438528:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00080000:3.0:1761313236.721948:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313236.721961:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313236.721971:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307438528:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761313236.722300:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307438336:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313236.722317:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307438464:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313236.722327:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307438400:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313236.963473:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313237.026930:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313237.985108:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313238.049587:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313239.009348:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313239.074327:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313240.032755:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313240.096831:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313241.056863:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313241.120836:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761313241.825000:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313241.825008:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761313241.825011:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1761313241.825019:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e662bceb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307438592:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761313241.825020:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1484 00000100:00080000:2.0:1761313241.825025:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 11s 00000100:00080000:2.0:1761313241.825032:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761313241.825042:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761313241.825096:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000015f165e6 x1846869307438784/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761313241.825125:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313241.825129:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761313241.825135:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002292ab57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307438720:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761313241.825153:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e3466f5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307438656:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313241.825279:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000015f165e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307438784:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761313241.834993:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e662bceb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307438592:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313241.835011:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002292ab57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307438720:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761313241.836349:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761313241.836356:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761313241.836364:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000015f165e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307438784:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761313241.836754:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e3466f5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307438656:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313242.080755:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313242.146195:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313243.105576:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313243.168882:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313244.128816:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313244.194139:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313245.152798:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313245.216908:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313246.176776:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313246.241815:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761313246.944876:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313246.944882:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761313246.944885:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761313246.944894:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1489 00000100:00080000:3.0:1761313246.944897:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 16s 00000100:00080000:3.0:1761313246.944903:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761313246.944911:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761313246.944925:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000004505177d x1846869307439040/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761313246.944961:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313246.944965:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761313246.944991:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cdcc674d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307438912:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313246.945082:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1761313246.945089:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307438848:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313246.945113:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307438976:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313246.945505:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307439040:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761313246.951981:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cdcc674d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307438912:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313246.962023:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307438848:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313246.962055:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307438976:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761313246.964478:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761313246.964488:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761313246.964497:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307439040:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761313247.205719:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313247.268752:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313248.224848:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313248.294652:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313249.248793:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313249.312835:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313250.275091:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313250.336866:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313251.297991:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313251.360907:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761313252.064921:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313252.064928:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761313252.064932:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761313252.064942:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1495 00000100:00080000:0.0:1761313252.064946:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761313252.064952:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761313252.064961:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761313252.064977:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000bcaee194 x1846869307439296/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761313252.065001:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313252.065005:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313252.065032:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307439168:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313252.065123:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1761313252.065129:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307439104:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313252.065151:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307439232:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313252.065395:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307439296:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761313252.076956:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307439104:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313252.076982:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307439232:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313252.076993:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307439168:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761313252.077034:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313252.077039:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313252.077045:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307439296:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761313252.320773:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313252.384950:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313253.344787:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313253.409666:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313254.371248:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313254.433159:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313255.392812:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313255.458139:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313256.416808:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313256.480825:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761313257.186326:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313257.186335:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761313257.186339:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761313257.186353:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1500 00000100:00080000:3.0:1761313257.186358:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761313257.186366:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761313257.186388:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761313257.186408:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006e93398e x1846869307439552/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761313257.186430:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313257.186434:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313257.186589:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307439552:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00080000:0.0:1761313257.188444:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761313257.188455:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761313257.188466:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307439552:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761313257.189769:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307439360:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313257.189868:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cdcc674d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307439488:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313257.189896:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761313257.189901:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307439424:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313257.197107:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307439360:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313257.199545:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307439424:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313257.199567:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cdcc674d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307439488:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313257.440841:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313257.505072:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313258.464874:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313258.528924:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313259.489462:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313259.553354:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313260.512766:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313260.577344:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313261.537939:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313261.601901:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761313262.305684:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307439616:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313262.309497:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307439616:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761313262.309865:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313262.309869:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761313262.309872:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761313262.309878:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1505 00000100:00080000:1.0:1761313262.309880:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1761313262.309886:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761313262.309894:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761313262.309908:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a15ef099 x1846869307439808/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761313262.309930:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313262.309933:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313262.310000:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307439680:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313262.310083:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761313262.310090:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307439744:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313262.310174:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307439808:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00080000:3.0:1761313262.313183:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313262.313193:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313262.313203:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307439808:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761313262.313301:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307439744:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313262.313316:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307439680:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313262.560770:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313262.626100:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313263.585168:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313263.650715:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313264.610938:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313264.674222:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313265.632763:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313265.697669:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313266.657085:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313266.721004:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761313267.424966:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313267.424976:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761313267.424980:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761313267.425019:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1510 00000100:00080000:3.0:1761313267.425023:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761313267.425031:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761313267.425046:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761313267.425067:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000602b09d0 x1846869307440064/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761313267.425096:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313267.425099:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313267.425193:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307440064:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761313267.425518:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004e00d332 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307439872:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313267.426066:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f133bb2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307440000:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313267.426557:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761313267.426569:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307439936:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313267.432971:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307439936:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313267.433020:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004e00d332 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307439872:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761313267.433032:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:2.0:1761313267.433038:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f133bb2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307440000:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761313267.433040:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313267.433049:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307440064:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761313267.680783:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313267.745172:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313268.704894:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313268.768917:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313269.728783:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313269.795691:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313270.752973:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313270.819547:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313271.776887:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313271.841062:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761313272.547425:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313272.547433:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761313272.547440:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761313272.547451:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1515 00000100:00080000:0.0:1761313272.547454:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761313272.547460:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761313272.547469:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761313272.547484:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a24c5c6a x1846869307440320/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761313272.547501:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313272.547504:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313272.547527:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307440192:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313272.547540:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307440320:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761313272.547622:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1761313272.547630:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307440128:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313272.547675:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307440256:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313272.551305:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307440128:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313272.551326:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307440256:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313272.551335:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307440192:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761313272.552548:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313272.552564:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313272.552571:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307440320:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761313272.801870:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313272.865203:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313273.824768:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313273.892584:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313274.848772:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313274.912897:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313275.873329:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313275.937545:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313276.896934:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313276.960881:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761313277.664839:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313277.664845:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761313277.664848:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761313277.664868:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1520 00000100:00080000:0.0:1761313277.664871:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761313277.664876:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761313277.664884:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761313277.664904:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a24c5c6a x1846869307440576/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761313277.664925:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313277.664928:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761313277.665003:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307440576:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761313277.666273:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307440448:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313277.666337:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1761313277.666343:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307440384:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313277.666363:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307440512:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761313277.669231:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313277.669241:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313277.669250:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307440576:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761313277.675724:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307440384:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313277.675746:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307440512:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313277.675755:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307440448:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313277.922967:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313277.986242:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313278.946131:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313279.009705:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313279.969034:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313280.033149:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313280.994290:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313281.057764:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313282.016798:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313282.080876:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761313282.784880:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313282.784888:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761313282.784891:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761313282.784902:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1525 00000100:00080000:2.0:1761313282.784905:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761313282.784911:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761313282.784921:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761313282.784937:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007ffb03e6 x1846869307440832/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761313282.784965:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313282.784968:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761313282.784990:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e3466f5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307440640:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761313282.785000:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039978a4e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307440704:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313282.785034:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ffb03e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307440832:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761313282.785708:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008a641e2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307440768:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761313282.788059:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039978a4e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307440704:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313282.788224:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e3466f5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307440640:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761313282.788301:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761313282.788306:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761313282.788312:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ffb03e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307440832:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761313282.788731:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008a641e2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307440768:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313283.041019:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313283.104883:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313284.064769:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313284.129049:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313285.088760:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313285.153210:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313286.112775:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313286.177880:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313287.136769:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313287.202371:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761313287.904832:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313287.904839:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761313287.904842:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761313287.904853:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1530 00000100:00080000:2.0:1761313287.904856:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761313287.904862:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761313287.904887:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761313287.904905:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000eca34e1c x1846869307441088/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761313287.904931:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313287.904934:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761313287.904990:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eca34e1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307441088:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761313287.905026:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039978a4e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307440896:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313287.905086:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ae09c97d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307441024:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313287.905117:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761313287.905124:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aea3302e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307440960:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761313287.908086:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761313287.908101:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761313287.908114:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eca34e1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307441088:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761313287.912410:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aea3302e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307440960:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313287.912436:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039978a4e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307440896:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313287.912459:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ae09c97d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307441024:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313288.161032:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313288.224943:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313289.185118:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313289.249570:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313290.208902:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313290.273766:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313291.232759:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313291.297607:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313292.256759:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313292.321165:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761313293.025949:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313293.025956:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761313293.025959:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761313293.025969:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1535 00000100:00080000:2.0:1761313293.025972:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761313293.025978:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761313293.025988:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761313293.026025:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e9376149 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307441152:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313293.026160:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ba5c8e77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307441280:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313293.026201:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761313293.026208:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b2298797 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307441216:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761313293.026833:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b4eac89a x1846869307441344/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761313293.026907:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b4eac89a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307441344:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00080000:2.0:1761313293.029658:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313293.029668:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761313293.031384:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b2298797 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307441216:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313293.031402:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ba5c8e77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307441280:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313293.033371:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e9376149 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307441152:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761313293.034424:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761313293.034432:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761313293.034438:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b4eac89a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307441344:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761313293.280760:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313293.344859:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313294.306897:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313294.369974:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313295.328752:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313295.393287:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313296.353101:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313296.416967:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313297.376756:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313297.440850:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761313298.144850:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313298.144857:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761313298.144860:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761313298.144870:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1541 00000100:00080000:2.0:1761313298.144874:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761313298.144880:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000100:00100000:3.0:1761313298.144888:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f220504f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307441408:192.168.203.151@tcp:400:kworker/u8:3.0 00000008:00100000:2.0:1761313298.144889:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761313298.144917:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000001426be00 x1846869307441600/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761313298.144938:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313298.144942:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761313298.144976:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001426be00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307441600:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761313298.144982:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000039eebc2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307441472:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313298.145003:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004592c0a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307441536:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761313298.154859:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000039eebc2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307441472:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761313298.154914:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f220504f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307441408:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761313298.154925:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004592c0a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307441536:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761313298.156685:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761313298.156693:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761313298.156701:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001426be00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307441600:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761313298.400764:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313298.467680:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313299.424727:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313299.490353:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313300.448848:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313300.514045:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313301.473682:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313301.537403:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313302.497047:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313302.560854:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761313303.270545:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313303.270552:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761313303.270556:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761313303.270567:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1546 00000100:00080000:0.0:1761313303.270570:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761313303.270576:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761313303.270585:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761313303.270614:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000bcaee194 x1846869307441856/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761313303.270691:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313303.270694:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313303.270719:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307441728:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313303.270802:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1761313303.270808:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307441664:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313303.270831:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307441792:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313303.271048:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307441856:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00080000:2.0:1761313303.280768:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761313303.280779:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761313303.280788:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307441856:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761313303.281483:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307441664:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313303.281500:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307441792:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313303.281510:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307441728:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313303.520921:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313303.585019:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313304.544767:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313304.610130:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313305.568763:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313305.633935:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313306.593345:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313306.657885:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313307.619688:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313307.681127:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761313308.385204:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313308.385213:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761313308.385217:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761313308.385230:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1551 00000100:00080000:3.0:1761313308.385234:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761313308.385241:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761313308.385252:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761313308.385269:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000039277a44 x1846869307442112/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761313308.385326:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313308.385330:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761313308.385546:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307441920:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761313308.385694:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004e00d332 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307442048:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761313308.385733:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761313308.385739:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f133bb2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307441984:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313308.388353:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307442112:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761313308.392255:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307441920:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761313308.392282:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004e00d332 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307442048:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313308.397966:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f133bb2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307441984:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761313308.398111:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761313308.398132:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761313308.398138:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307442112:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761313308.656634:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313308.705346:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313309.664765:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313309.728858:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313310.688798:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313310.754685:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313311.712797:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313311.778348:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313312.736765:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313312.801306:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313313.504936:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307442176:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313313.507286:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307442240:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761313313.509188:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1761313313.509359:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761313313.509380:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307442304:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761313313.510374:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761313313.510378:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761313313.510386:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1556 00000100:00080000:0.0:1761313313.510389:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761313313.510396:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761313313.510409:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761313313.510439:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007dcceb4a x1846869307442368/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761313313.510484:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313313.510488:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761313313.510624:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307442368:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761313313.513853:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307442304:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313313.513874:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307442240:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313313.516545:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307442176:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761313313.518938:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761313313.518951:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761313313.518961:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307442368:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761313313.760794:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313313.825795:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313314.784761:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313314.848876:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313315.808825:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313315.873127:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313316.835889:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313316.896874:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313317.857763:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313317.922603:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761313318.624956:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313318.624985:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761313318.624989:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761313318.625002:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1561 00000100:00080000:3.0:1761313318.625006:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761313318.625015:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761313318.625026:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761313318.625057:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ddd3c50a x1846869307442624/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761313318.625136:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307442624:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761313318.625682:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f133bb2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307442432:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761313318.626286:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307442560:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761313318.626328:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761313318.626334:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cdcc674d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307442496:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761313318.627333:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313318.627340:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761313318.639118:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307442560:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761313318.639882:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761313318.639889:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761313318.639895:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307442624:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761313318.639979:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cdcc674d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307442496:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761313318.639991:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f133bb2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307442432:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313318.880913:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313318.944907:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313319.904802:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313319.969115:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313320.928803:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313320.994303:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313321.952703:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313322.017432:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313322.977096:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313323.040953:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761313323.744954:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307442688:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761313323.746548:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313323.746556:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761313323.746560:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761313323.746570:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1566 00000100:00080000:1.0:1761313323.746574:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1761313323.746580:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761313323.746593:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761313323.746609:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a1719d14 x1846869307442880/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761313323.746630:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1761313323.746715:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307442752:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313323.746780:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761313323.746787:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307442816:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313323.746956:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307442880:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00080000:1.0:1761313323.747455:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761313323.752901:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307442816:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313323.752929:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307442752:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313323.757505:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307442688:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761313323.758699:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761313323.758710:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761313323.758718:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307442880:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761313324.004162:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313324.064958:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313325.025171:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313325.090323:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313326.048855:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313326.113530:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313327.072765:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313327.137837:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313328.096774:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313328.161183:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761313328.864916:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307442944:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313328.866929:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307443008:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313328.869417:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307442944:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313328.870369:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307443008:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761313328.871286:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313328.871294:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761313328.871297:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761313328.871305:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1571 00000100:00080000:1.0:1761313328.871308:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1761313328.871315:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761313328.871328:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761313328.871347:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000163e3801 x1846869307443136/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761313328.871370:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313328.871374:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761313328.871415:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761313328.871430:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307443072:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313328.871744:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307443136:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00080000:2.0:1761313328.878542:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761313328.878575:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761313328.878586:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307443136:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761313328.880770:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307443072:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313329.122114:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313329.186741:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313330.145831:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313330.210656:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313331.169025:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313331.234809:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313332.192765:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313332.260501:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313333.216823:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313333.281115:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761313333.985173:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313333.985202:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761313333.985206:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761313333.985219:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1576 00000100:00080000:0.0:1761313333.985223:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761313333.985231:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761313333.985243:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761313333.985264:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000034a6cd79 x1846869307443392/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761313333.985291:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313333.985295:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313333.985334:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307443264:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313333.985439:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1761313333.985446:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307443200:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313333.985471:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307443328:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761313333.985808:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307443392:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761313333.998764:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307443328:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313333.998788:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307443264:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313333.999834:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307443200:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761313334.000967:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761313334.000978:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761313334.000986:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307443392:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761313334.241374:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313334.304877:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313335.264791:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313335.328971:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313336.288811:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313336.353003:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313337.312801:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313337.387461:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313338.336771:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313338.400925:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761313339.104916:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313339.104924:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761313339.104927:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761313339.104938:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1582 00000100:00080000:1.0:1761313339.104941:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1761313339.104947:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761313339.104956:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761313339.104972:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000eb8b1f19 x1846869307443648/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761313339.105149:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313339.105153:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761313339.105182:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307443520:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313339.105264:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1761313339.105276:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307443456:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313339.105301:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307443584:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313339.112861:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307443456:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313339.113066:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307443648:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761313339.116407:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307443520:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313339.117324:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307443584:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761313339.120983:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761313339.120994:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761313339.121003:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307443648:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761313339.360882:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313339.425123:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313340.384862:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313340.448997:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313341.408770:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313341.473140:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313342.432789:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313342.496972:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313343.456767:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313343.520822:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761313344.225842:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313344.225852:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761313344.225857:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761313344.225869:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1587 00000100:00080000:1.0:1761313344.225873:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1761313344.225881:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761313344.225914:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761313344.225936:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a15ef099 x1846869307443904/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761313344.225964:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307443712:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761313344.225967:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313344.225973:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761313344.226022:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307443776:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313344.229105:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307443904:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761313344.229767:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307443840:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313344.236355:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307443776:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313344.236530:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307443712:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313344.236543:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307443840:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761313344.237036:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761313344.237051:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761313344.237057:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307443904:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761313344.480797:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313344.545485:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313345.504808:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313345.569270:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313346.528963:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313346.592927:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313347.553498:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313347.622243:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313348.580724:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313348.640833:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761313349.344883:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313349.344892:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761313349.344897:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761313349.344911:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1592 00000100:00080000:2.0:1761313349.344915:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761313349.344923:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761313349.344937:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761313349.344954:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004592c0a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307443968:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313349.344987:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b8c05a37 x1846869307444160/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761313349.345019:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313349.345023:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761313349.345054:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f220504f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307444032:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313349.345061:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307444160:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761313349.345096:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000039eebc2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307444096:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313349.351603:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004592c0a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307443968:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313349.352345:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f220504f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307444032:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313349.352392:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000039eebc2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307444096:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761313349.361771:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761313349.361785:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761313349.361796:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307444160:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761313349.600812:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313349.669808:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313350.624767:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313350.691934:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313351.648804:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313351.718167:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313352.672828:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313352.740324:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313353.696816:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313353.762680:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761313354.467689:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307444224:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761313354.469742:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313354.469752:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761313354.469756:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761313354.469769:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1597 00000100:00080000:1.0:1761313354.469792:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1761313354.469799:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761313354.469812:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761313354.469832:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a15ef099 x1846869307444416/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761313354.469855:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313354.469859:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761313354.469895:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307444352:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313354.469994:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761313354.470001:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307444288:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313354.470511:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307444416:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761313354.476031:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307444224:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313354.478105:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307444288:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313354.478126:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307444352:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761313354.478228:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761313354.478234:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761313354.478239:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307444416:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761313354.721182:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313354.786096:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313355.745280:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313355.808884:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313356.768844:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313356.832913:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313357.793144:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313357.858050:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313358.816762:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313358.881866:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761313359.585066:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313359.585072:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761313359.585075:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761313359.585086:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1602 00000100:00080000:0.0:1761313359.585089:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761313359.585095:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761313359.585104:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761313359.585118:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000084f3d917 x1846869307444672/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761313359.585200:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307444672:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761313359.585316:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307444480:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313359.585347:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307444608:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313359.585372:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761313359.585376:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307444544:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761313359.586461:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313359.586468:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00080000:2.0:1761313359.590576:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761313359.590585:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761313359.590594:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307444672:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761313359.590676:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307444544:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313359.590686:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307444480:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313359.590694:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307444608:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313359.840765:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313359.906214:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313360.864815:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313360.932724:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313361.888773:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313361.954430:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313362.912771:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313362.977539:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313363.938606:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313364.001193:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761313364.704917:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313364.704926:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761313364.704930:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761313364.704945:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1607 00000100:00080000:0.0:1761313364.704949:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761313364.704957:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761313364.704970:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761313364.704988:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307444736:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313364.704990:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000005a420586 x1846869307444928/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761313364.705086:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307444928:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761313364.705101:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307444864:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313364.705159:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761313364.705174:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307444800:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761313364.706659:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313364.706669:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313364.713478:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307444736:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313364.725157:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307444864:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313364.725216:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307444800:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761313364.725319:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761313364.725326:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761313364.725338:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307444928:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761313364.961869:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313365.026540:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313365.984884:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313366.051018:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313367.009687:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313367.073394:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313368.032780:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313368.097948:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313369.056768:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313369.121193:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313369.825628:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307444992:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761313369.827214:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313369.827223:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761313369.827227:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761313369.827237:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1612 00000100:00080000:0.0:1761313369.827240:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761313369.827247:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761313369.827259:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761313369.827278:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ace4d025 x1846869307445184/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761313369.827339:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307445184:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00080000:0.0:1761313369.828515:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313369.828523:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761313369.828721:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307445056:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313369.829126:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761313369.829135:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307445120:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313369.835622:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307445120:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313369.835677:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307445056:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313369.838883:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307444992:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:02020000:2.0:1761313369.839039:0:2262:0:(client.c:1361:ptlrpc_check_status()) 11-0: lustre-OST0000-osc-ffff951bc4d8c800: operation ost_connect to node 192.168.203.151@tcp failed: rc = -19 00000100:00080000:2.0:1761313369.847202:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761313369.847218:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761313369.847229:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307445184:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761313370.081970:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313370.145604:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313371.104749:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313371.168836:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313372.130882:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313372.192886:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313373.152746:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313373.217052:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313374.180381:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313374.240816:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761313374.944918:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313374.944939:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761313374.944942:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761313374.944951:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1617 00000100:00080000:1.0:1761313374.944954:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1761313374.944960:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761313374.944969:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761313374.944985:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000003871442b x1846869307445440/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761313374.945008:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313374.945011:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313374.945081:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307445248:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313374.945161:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307445376:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313374.945191:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761313374.945204:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307445312:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313374.945434:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307445440:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761313374.952939:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307445312:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313374.952953:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307445248:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313374.952961:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307445376:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761313374.954402:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761313374.954411:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761313374.954418:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307445440:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761313375.200895:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313375.264870:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313376.224728:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313376.288805:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313377.249304:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313377.315971:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313378.272770:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313378.336837:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313379.297153:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313379.361050:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761313380.064837:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313380.064845:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761313380.064848:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761313380.064860:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1623 00000100:00080000:1.0:1761313380.064863:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1761313380.064869:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000100:00100000:0.0:1761313380.064874:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307445504:192.168.203.151@tcp:400:kworker/u8:0.0 00000008:00100000:1.0:1761313380.064879:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761313380.064896:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000009cec3faa x1846869307445696/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761313380.064934:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313380.064938:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313380.064963:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307445632:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313380.064984:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307445696:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761313380.064998:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761313380.065007:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307445568:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761313380.070623:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761313380.070669:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761313380.070682:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307445696:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761313380.070780:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307445632:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313380.070813:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307445568:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313380.070827:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307445504:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313380.326483:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313380.388025:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313381.344819:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313381.409729:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313382.368820:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313382.434248:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313383.392834:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313383.458139:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313384.416829:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313384.480936:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313385.185056:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307445760:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313385.185543:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761313385.185552:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f220504f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307445824:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761313385.185863:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313385.185869:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761313385.185873:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761313385.185879:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1628 00000100:00080000:2.0:1761313385.185883:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761313385.185888:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761313385.185907:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761313385.185946:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000039eebc2 x1846869307445952/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1761313385.185963:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00080000:2.0:1761313385.185970:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313385.185976:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761313385.185976:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000007136630 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307445888:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313385.186072:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000039eebc2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307445952:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761313385.187001:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307445760:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313385.189872:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f220504f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307445824:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313385.193430:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000007136630 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307445888:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761313385.193738:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761313385.193744:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761313385.193750:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000039eebc2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307445952:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761313385.441079:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313385.505578:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313386.464814:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313386.530585:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313387.490366:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313387.554512:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313388.512793:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313388.576911:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313389.537433:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313389.601029:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761313390.304844:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313390.304853:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761313390.304857:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761313390.308354:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1633 00000100:00080000:0.0:1761313390.308364:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761313390.308399:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761313390.308416:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761313390.308456:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b5a39135 x1846869307446208/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761313390.308493:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313390.308496:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313390.308545:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307446080:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761313390.308546:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307446208:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761313390.308690:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1761313390.308699:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307446016:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313390.308743:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307446144:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313390.317874:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307446144:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313390.317906:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307446080:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761313390.321735:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761313390.321885:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761313390.321898:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307446208:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761313390.325156:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307446016:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313390.560813:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313390.626045:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313391.584789:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313391.648974:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313392.608834:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313392.673284:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313393.632856:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313393.696881:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313394.656825:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313394.723790:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313395.425111:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307446272:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761313395.430372:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307446336:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761313395.430372:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313395.430379:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761313395.430382:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761313395.430386:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1638 00000100:00080000:3.0:1761313395.430389:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761313395.430394:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761313395.430402:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761313395.430414:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000626f98db x1846869307446464/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761313395.430437:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313395.430440:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761313395.430453:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761313395.430458:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000101f852a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307446400:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313395.430671:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000626f98db pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307446464:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761313395.431012:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307446272:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761313395.435266:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307446336:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761313395.435323:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000101f852a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307446400:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761313395.435538:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761313395.435542:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761313395.435547:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000626f98db pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307446464:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761313395.683449:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313395.750042:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313396.705013:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313396.768877:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313397.728790:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313397.793671:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313398.752836:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313398.817387:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313399.776792:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313399.840885:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313400.546325:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307446528:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761313400.547727:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313400.547750:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761313400.547754:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761313400.547763:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1643 00000100:00080000:0.0:1761313400.547767:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761313400.547774:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761313400.547807:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761313400.547825:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000546b1bfe x1846869307446720/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761313400.547851:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313400.547854:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313400.547880:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307446592:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313400.547951:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761313400.547958:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307446656:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313400.549299:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307446720:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00080000:3.0:1761313400.562045:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313400.562058:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313400.562069:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307446720:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761313400.562238:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307446656:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313400.562252:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307446592:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313400.562285:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307446528:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313400.800821:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313400.865914:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313401.824760:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313401.888911:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313402.849067:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313402.912940:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313403.872802:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313403.939179:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313404.896807:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313404.960924:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761313405.664897:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313405.664908:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761313405.664912:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761313405.664926:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1648 00000100:00080000:0.0:1761313405.664930:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761313405.664938:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761313405.664951:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761313405.664972:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000546b1bfe x1846869307446976/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761313405.664997:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313405.665021:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313405.665088:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307446848:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313405.665214:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1761313405.665221:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307446784:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313405.665247:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307446912:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313405.669631:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307446976:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761313405.670713:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307446912:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313405.670733:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307446848:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313405.674029:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307446784:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761313405.678442:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313405.678462:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313405.678477:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307446976:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761313405.920789:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313405.984867:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313406.944823:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313407.010444:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313407.968800:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313408.034391:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313408.992811:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313409.060173:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313410.016796:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313410.080911:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761313410.786154:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313410.786163:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761313410.786166:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761313410.786179:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1653 00000100:00080000:2.0:1761313410.786182:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761313410.786189:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761313410.786200:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761313410.786216:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b8c05a37 x1846869307447232/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761313410.786231:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313410.786234:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313410.786323:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307447232:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761313410.786591:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000039eebc2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307447040:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313410.787466:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000007136630 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307447104:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313410.789840:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f220504f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307447168:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313410.795071:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f220504f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307447168:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313410.797811:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000007136630 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307447104:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313410.801059:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000039eebc2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307447040:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761313410.801157:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761313410.801163:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761313410.801168:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307447232:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761313411.040787:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313411.105051:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313412.065126:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313412.129045:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313413.088816:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313413.153497:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313414.113360:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313414.180477:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313415.136809:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313415.201236:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761313415.906334:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313415.906360:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761313415.906365:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761313415.906377:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1658 00000100:00080000:3.0:1761313415.906381:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761313415.906388:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761313415.906403:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761313415.906422:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000bfffbafe x1846869307447488/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761313415.906451:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313415.906455:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761313415.906491:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307447360:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313415.906668:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1761313415.906676:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307447296:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313415.906705:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000038534f0b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307447424:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313415.906857:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307447488:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00080000:0.0:1761313415.909181:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761313415.909194:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761313415.909205:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307447488:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761313415.909542:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000038534f0b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307447424:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313415.909557:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307447360:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313415.911735:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307447296:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313416.162883:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313416.225426:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313417.185230:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313417.248886:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313418.208834:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313418.272932:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313419.233158:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313419.299959:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313420.262582:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313420.320897:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761313421.024889:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313421.024900:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761313421.024920:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761313421.024934:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1663 00000100:00080000:3.0:1761313421.024938:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761313421.024945:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761313421.024958:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761313421.024979:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000d82b82f x1846869307447744/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761313421.025009:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313421.025013:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761313421.025047:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307447616:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313421.025171:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1761313421.025178:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307447552:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313421.025206:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000038534f0b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307447680:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313421.026182:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307447744:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761313421.030067:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307447552:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313421.033798:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307447616:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761313421.034025:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761313421.034030:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761313421.034035:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307447744:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761313421.034125:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000038534f0b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307447680:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313421.283821:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313421.345256:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313422.304788:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313422.368964:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313423.332456:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313423.392812:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313424.352854:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313424.417866:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313425.378100:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313425.442913:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313426.145028:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307447808:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313426.145836:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761313426.145847:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307447872:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761313426.145893:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313426.145899:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761313426.145903:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761313426.145909:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1669 00000100:00080000:0.0:1761313426.145912:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761313426.145918:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761313426.145931:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761313426.145950:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008e59d025 x1846869307448000/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761313426.145975:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313426.145979:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313426.146014:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307447936:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313426.146126:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307448000:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00080000:3.0:1761313426.150305:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313426.150332:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313426.150343:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307448000:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761313426.150426:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307447936:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313426.150440:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307447872:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313426.150603:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307447808:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313426.400804:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313426.465194:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313427.424798:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313427.491944:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313428.448805:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313428.514736:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313429.473084:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313429.538481:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313430.496779:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313430.563014:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761313431.264908:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313431.264918:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761313431.264922:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761313431.264935:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1674 00000100:00080000:1.0:1761313431.264939:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761313431.264946:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761313431.264959:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761313431.264979:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000163e3801 x1846869307448256/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761313431.265005:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313431.265008:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761313431.265055:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307448256:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761313431.265105:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307448064:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313431.265179:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307448192:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313431.265213:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761313431.265220:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307448128:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313431.270960:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307448128:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313431.270986:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307448064:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313431.270997:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307448192:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761313431.276780:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313431.276793:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313431.276804:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307448256:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761313431.521038:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313431.585336:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313432.544795:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313432.612018:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313433.568786:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313433.632951:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313434.593558:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313434.657880:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313435.618692:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313435.681159:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761313436.384856:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313436.384864:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761313436.384879:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761313436.384891:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1679 00000100:00080000:1.0:1761313436.384894:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761313436.384901:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761313436.384911:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761313436.384928:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ed2f9bca x1846869307448512/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761313436.384953:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313436.384956:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761313436.385064:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307448512:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761313436.385065:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307448320:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313436.385190:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307448448:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313436.385398:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761313436.385408:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307448384:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313436.396329:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307448384:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313436.396345:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307448320:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313436.396354:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307448448:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761313436.396481:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313436.396486:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313436.396491:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307448512:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761313436.640800:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313436.706089:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313437.664817:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313437.728908:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313438.688818:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313438.753602:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313439.713687:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313439.779475:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313440.738670:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313440.803016:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313441.504882:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307448576:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761313441.505371:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313441.505376:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761313441.505379:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761313441.505385:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1684 00000100:00080000:0.0:1761313441.505388:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761313441.505393:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761313441.505403:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761313441.505417:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b5a39135 x1846869307448768/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1761313441.505489:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307448768:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761313441.506679:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307448640:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313441.506849:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761313441.506856:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307448704:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761313441.507425:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313441.507430:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00080000:3.0:1761313441.511517:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313441.511528:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313441.511537:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307448768:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761313441.516124:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307448640:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313441.516481:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307448576:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313441.517731:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307448704:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313441.760800:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313441.825533:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313442.784760:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313442.850667:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313443.809087:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313443.872969:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313444.836462:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313444.896909:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313445.856767:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313445.924210:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761313446.627433:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313446.627440:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761313446.627443:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761313446.627454:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1689 00000100:00080000:1.0:1761313446.627457:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761313446.627462:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761313446.627483:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761313446.627498:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000009cec3faa x1846869307449024/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761313446.627523:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313446.627535:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761313446.627562:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307448896:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313446.627668:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1761313446.627674:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307448832:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313446.627695:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307448960:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313446.628402:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307449024:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00080000:3.0:1761313446.634549:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313446.634564:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313446.634576:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307449024:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761313446.634795:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307448832:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313446.634813:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307448960:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313446.634821:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307448896:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313446.880800:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313446.944897:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313447.904828:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313447.969761:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313448.929290:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313448.994537:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313449.954556:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313450.016839:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313450.976805:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313451.040846:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313451.749714:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000007136630 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307449088:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761313451.749736:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313451.749746:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761313451.749750:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761313451.749757:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1694 00000100:00080000:2.0:1761313451.749768:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761313451.749775:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761313451.749788:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761313451.749808:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b4eac89a x1846869307449280/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761313451.749841:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313451.749845:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313451.749885:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b4eac89a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307449280:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761313451.749888:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004592c0a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307449152:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313451.751165:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761313451.751182:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001426be00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307449216:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761313451.753512:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313451.753525:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313451.753537:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b4eac89a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307449280:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761313451.757855:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000007136630 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307449088:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313451.760219:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001426be00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307449216:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313451.760247:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004592c0a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307449152:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313452.001463:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313452.065269:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313453.027706:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313453.089457:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313454.052546:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313454.112899:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313455.074704:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313455.140471:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313456.096822:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313456.161997:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761313456.867363:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313456.867374:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761313456.867378:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761313456.867391:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1699 00000100:00080000:0.0:1761313456.867396:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00100000:1.0:1761313456.867400:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307449344:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761313456.867406:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761313456.867422:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761313456.867441:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000005a420586 x1846869307449536/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761313456.867468:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313456.867472:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313456.867499:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307449408:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313456.867526:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307449472:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313456.867695:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307449536:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761313456.872287:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307449344:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313456.873193:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307449472:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761313456.873354:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313456.873361:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313456.873366:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307449536:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761313456.873454:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307449408:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313457.121833:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313457.184935:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313458.144810:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313458.212517:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313459.168780:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313459.232864:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313460.194455:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313460.258417:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313461.216795:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313461.280931:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761313461.984849:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313461.984858:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761313461.984862:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761313461.984874:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1704 00000100:00080000:2.0:1761313461.984878:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761313461.984884:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761313461.984897:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761313461.984914:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ba5c8e77 x1846869307449792/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761313461.984972:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ba5c8e77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307449792:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761313461.985142:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004592c0a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307449600:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313461.985183:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e9376149 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307449728:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313461.985240:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761313461.985267:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001426be00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307449664:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761313461.986403:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313461.986420:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00080000:0.0:1761313461.986836:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761313461.986842:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761313461.986860:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ba5c8e77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307449792:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761313461.987720:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004592c0a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307449600:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313461.993525:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001426be00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307449664:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313461.993549:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e9376149 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307449728:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313462.240808:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313462.304955:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313463.264798:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313463.328900:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313464.297184:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313464.352900:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313465.313289:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313465.376915:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313466.337564:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313466.405557:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313467.105684:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307449856:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761313467.106182:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313467.106189:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761313467.106193:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761313467.106200:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1710 00000100:00080000:0.0:1761313467.106204:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761313467.106210:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761313467.106223:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761313467.106241:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000034a6cd79 x1846869307450048/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1761313467.106310:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307450048:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761313467.108165:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307449920:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313467.108258:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761313467.108284:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307449984:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761313467.109111:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313467.109118:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313467.111545:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307449856:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313467.114267:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307449984:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313467.114292:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307449920:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761313467.114987:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313467.114994:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313467.115001:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307450048:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761313467.361143:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313467.424984:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313468.385153:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313468.449519:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313469.409248:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313469.473382:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313470.433184:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313470.496938:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313471.456771:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313471.521205:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313472.225372:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000038534f0b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307450112:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313472.226746:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307450176:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313472.228426:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00080000:3.0:1761313472.228439:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1761313472.228442:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307450240:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761313472.228447:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761313472.228451:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761313472.228459:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1715 00000100:00080000:3.0:1761313472.228476:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761313472.228482:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761313472.228495:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761313472.228516:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a183f7ef x1846869307450304/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761313472.228544:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313472.228549:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313472.228941:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307450304:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761313472.231610:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000038534f0b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307450112:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313472.231722:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307450176:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313472.234712:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307450240:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761313472.234926:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313472.234930:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313472.234935:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307450304:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761313472.480846:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313472.547169:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313473.504793:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313473.568854:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313474.528947:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313474.592922:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313475.552803:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313475.616907:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313476.577524:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313476.644366:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761313477.345830:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313477.345845:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761313477.345849:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761313477.345863:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1720 00000100:00080000:1.0:1761313477.345867:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761313477.345873:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761313477.345885:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761313477.345908:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000009cec3faa x1846869307450560/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761313477.345934:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313477.345937:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761313477.346027:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307450560:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761313477.346071:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307450368:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313477.346209:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307450496:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313477.346244:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761313477.346250:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307450432:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313477.350923:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307450432:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313477.350944:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307450496:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761313477.351730:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313477.351737:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313477.351745:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307450560:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761313477.352385:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307450368:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313477.601296:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313477.664951:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313478.625349:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313478.690958:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313479.649751:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313479.713462:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313480.672806:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313480.738319:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313481.696805:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313481.760790:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761313482.464889:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313482.464900:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761313482.464905:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761313482.464920:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1725 00000100:00080000:0.0:1761313482.464924:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761313482.464931:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761313482.464944:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761313482.464963:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007dcceb4a x1846869307450816/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1761313482.465037:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307450816:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761313482.465574:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307450624:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313482.465625:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307450752:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313482.465681:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761313482.465688:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307450688:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761313482.465984:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313482.465990:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313482.476344:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307450688:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313482.476371:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307450752:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313482.479368:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307450624:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761313482.480038:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313482.480045:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313482.480052:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307450816:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761313482.720910:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313482.786834:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313483.744913:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313483.810782:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313484.768796:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313484.834148:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313485.792799:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313485.859873:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313486.816734:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313486.884687:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761313487.586611:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313487.586621:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761313487.586625:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761313487.586666:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1730 00000100:00080000:0.0:1761313487.586670:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00100000:1.0:1761313487.586676:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307450880:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761313487.586677:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761313487.586689:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761313487.586707:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000fe4020c4 x1846869307451072/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761313487.586732:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313487.586735:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313487.586760:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307450944:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313487.586775:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307451008:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313487.587132:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307451072:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761313487.591180:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307450880:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313487.591211:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307451008:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313487.591449:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307450944:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761313487.594105:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313487.594119:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313487.594131:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307451072:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761313487.840774:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313487.905821:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313488.867355:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313488.928882:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313489.888814:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313489.952880:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313490.912900:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313490.978531:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313491.936818:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313492.000918:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761313492.705119:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313492.705130:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761313492.705134:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761313492.705147:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1735 00000100:00080000:2.0:1761313492.705151:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761313492.705159:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761313492.705172:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761313492.705192:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000aea3302e x1846869307451328/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761313492.705218:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313492.705221:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761313492.705260:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ae09c97d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307451200:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761313492.705362:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1761313492.705367:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b2298797 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307451136:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761313492.705386:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039978a4e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307451264:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313492.705493:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aea3302e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307451328:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761313492.710574:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b2298797 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307451136:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761313492.711261:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039978a4e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307451264:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761313492.711274:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ae09c97d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307451200:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761313492.712103:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313492.712109:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313492.712117:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aea3302e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307451328:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761313492.960776:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313493.026148:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313493.984800:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313494.050778:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313495.011237:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313495.075002:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313496.033443:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313496.097960:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313497.058023:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313497.122394:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761313497.825106:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313497.825117:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761313497.825121:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761313497.825135:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1740 00000100:00080000:0.0:1761313497.825140:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00100000:1.0:1761313497.825145:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307451392:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761313497.825150:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761313497.825165:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761313497.825211:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000028de4b9d x1846869307451584/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761313497.825237:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313497.825242:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313497.825282:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307451456:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313497.825282:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307451584:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761313497.826360:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307451520:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761313497.833478:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313497.833492:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313497.833505:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307451584:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761313497.833599:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307451456:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313497.834351:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307451392:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313497.834368:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307451520:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313498.080978:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313498.145692:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313499.105491:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313499.170557:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313500.128924:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313500.195263:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313501.155194:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313501.216865:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313502.176814:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313502.240859:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761313502.945292:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313502.945301:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761313502.945306:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761313502.945319:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1745 00000100:00080000:3.0:1761313502.945342:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761313502.945350:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761313502.945363:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761313502.945384:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f597cda6 x1846869307451840/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761313502.945416:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313502.945420:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761313502.945456:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307451712:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313502.945469:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f597cda6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307451840:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761313502.945586:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307451648:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761313502.945760:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307451776:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313502.949234:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307451712:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761313502.949865:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761313502.949872:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761313502.949879:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f597cda6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307451840:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761313502.950032:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307451776:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761313502.952018:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307451648:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313503.201184:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313503.267798:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313504.224794:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313504.288922:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313505.250959:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313505.313204:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313506.273099:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313506.339238:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313507.298177:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313507.361056:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313508.065132:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307451904:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761313508.066068:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cc6dd8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307452032:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761313508.066136:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761313508.066152:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007673f690 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307451968:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761313508.066466:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313508.066474:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761313508.066478:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761313508.066484:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1751 00000100:00080000:3.0:1761313508.066487:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761313508.066494:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761313508.066507:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761313508.066527:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000009911ad3 x1846869307452096/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761313508.066577:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313508.066580:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761313508.066650:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000009911ad3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307452096:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761313508.072334:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307451904:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761313508.072971:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313508.072977:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313508.072983:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000009911ad3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307452096:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761313508.076956:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007673f690 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307451968:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761313508.077002:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc6dd8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307452032:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313508.320927:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313508.385138:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313509.344739:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313509.409501:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313510.369126:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313510.433773:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313511.393317:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313511.458195:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313512.416788:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313512.481688:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761313513.188429:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313513.188440:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761313513.188444:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761313513.188457:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1756 00000100:00080000:0.0:1761313513.188462:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761313513.188469:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761313513.188482:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761313513.188502:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a24c5c6a x1846869307452352/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761313513.188525:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313513.188528:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313513.188553:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307452224:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313513.188684:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1761313513.188699:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307452160:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313513.188728:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307452288:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313513.188801:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307452352:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00080000:3.0:1761313513.191055:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313513.191071:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313513.191084:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307452352:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761313513.191179:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307452160:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313513.192140:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307452288:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313513.192157:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307452224:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313513.444277:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313513.505008:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313514.464936:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313514.529146:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313515.490946:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313515.553398:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313516.515880:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313516.577959:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313517.539470:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313517.604080:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313518.305927:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307452416:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761313518.309011:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313518.309020:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761313518.309024:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761313518.309037:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1761 00000100:00080000:0.0:1761313518.309041:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00100000:1.0:1761313518.309345:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307452480:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313518.309451:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761313518.309457:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307452544:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313518.310579:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307452416:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761313518.310692:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761313518.310708:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761313518.310738:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a24c5c6a x1846869307452608/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1761313518.310798:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307452608:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00080000:0.0:1761313518.312067:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313518.312079:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313518.323418:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307452544:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313518.323444:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307452480:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761313518.329144:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313518.329161:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313518.329176:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307452608:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761313518.560876:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313518.625335:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313519.585227:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313519.650347:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313520.608770:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313520.673135:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313521.632762:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313521.699404:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313522.656770:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313522.721881:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761313523.424869:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313523.424881:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761313523.424885:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761313523.424896:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1766 00000100:00080000:0.0:1761313523.424913:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761313523.424920:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761313523.424929:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761313523.424946:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000028de4b9d x1846869307452864/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1761313523.425019:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307452864:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00080000:0.0:1761313523.425915:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313523.425920:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761313523.425994:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307452672:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313523.426037:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307452800:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313523.426075:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761313523.426081:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307452736:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313523.433079:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307452800:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313523.433114:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307452672:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:02020000:3.0:1761313523.433277:0:2262:0:(client.c:1361:ptlrpc_check_status()) 11-0: lustre-OST0000-osc-ffff951bc4d8c800: operation ost_connect to node 192.168.203.151@tcp failed: rc = -19 00000100:00100000:0.0:1761313523.433898:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307452736:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761313523.441135:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313523.441151:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313523.441166:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307452864:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761313523.685368:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313523.745424:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313524.704859:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313524.770280:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313525.730059:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313525.792942:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313526.752823:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313526.818861:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313527.777058:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313527.844621:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761313528.545133:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313528.545144:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761313528.545149:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761313528.545164:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1771 00000100:00080000:2.0:1761313528.545168:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761313528.545175:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761313528.545188:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761313528.545209:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000eca34e1c x1846869307453120/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761313528.545235:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313528.545239:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313528.545273:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eca34e1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307453120:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761313528.545430:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ae09c97d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307452928:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313528.545474:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b2298797 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307453056:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313528.545502:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761313528.545507:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039978a4e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307452992:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313528.549956:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ae09c97d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307452928:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313528.551556:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039978a4e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307452992:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313528.551609:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b2298797 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307453056:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761313528.554233:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761313528.554247:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761313528.554259:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eca34e1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307453120:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761313528.801130:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313528.865969:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313529.824765:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313529.892217:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313530.852320:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313530.913033:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313531.872762:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313531.936903:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313532.897475:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313532.962741:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761313533.665027:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313533.665033:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761313533.665036:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761313533.665046:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1776 00000100:00080000:0.0:1761313533.665049:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761313533.665055:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761313533.665065:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761313533.665095:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000fe4020c4 x1846869307453376/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761313533.665122:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313533.665125:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313533.665151:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307453248:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313533.665230:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1761313533.665236:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307453184:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313533.665260:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307453312:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313533.665489:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307453376:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761313533.666815:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307453248:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313533.668539:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307453184:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313533.668560:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307453312:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761313533.674988:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313533.674998:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313533.675008:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307453376:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761313533.920764:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313533.984847:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313534.944776:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313535.008873:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313535.968799:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313536.033034:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313536.993195:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313537.059262:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313538.016816:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313538.082889:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761313538.787919:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313538.787928:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761313538.787932:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761313538.787945:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1781 00000100:00080000:0.0:1761313538.787949:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761313538.787955:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761313538.787966:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761313538.787982:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000034a6cd79 x1846869307453632/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761313538.788007:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313538.788011:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313538.788045:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307453504:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313538.789058:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1761313538.789072:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307453440:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313538.789123:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307453568:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313538.791778:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307453632:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761313538.791887:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307453440:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313538.791910:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307453568:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313538.791923:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307453504:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761313538.792936:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313538.792944:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313538.792951:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307453632:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761313539.040797:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313539.106426:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313540.064809:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313540.130167:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313541.088983:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313541.152912:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313542.113342:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313542.178965:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313543.136800:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313543.200891:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313543.904973:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008a641e2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307453696:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761313543.906042:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313543.906047:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761313543.906051:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761313543.906058:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1786 00000100:00080000:2.0:1761313543.906060:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761313543.906065:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761313543.906074:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761313543.906088:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000015f165e6 x1846869307453888/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761313543.906145:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000015f165e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307453888:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761313543.906312:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e3466f5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307453824:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313543.906344:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761313543.906350:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ffb03e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307453760:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761313543.906820:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313543.906825:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761313543.909971:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ffb03e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307453760:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313543.909990:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e3466f5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307453824:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313543.910000:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008a641e2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307453696:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761313543.912451:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761313543.912460:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761313543.912469:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000015f165e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307453888:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761313544.160774:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313544.224945:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313545.184797:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313545.248870:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313546.208766:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313546.272805:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313547.232793:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313547.296877:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313548.257572:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313548.321963:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761313549.024892:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313549.024898:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761313549.024901:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761313549.024911:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1791 00000100:00080000:3.0:1761313549.024914:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761313549.024919:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761313549.024930:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761313549.024943:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f364ec74 x1846869307454144/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761313549.024966:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313549.024969:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761313549.024992:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e58cc722 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307454016:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313549.025077:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1761313549.025083:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000009911ad3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307453952:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313549.025103:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307454080:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313549.025381:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f364ec74 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307454144:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761313549.031381:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e58cc722 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307454016:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313549.033184:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000009911ad3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307453952:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313549.033206:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307454080:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761313549.034266:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761313549.034273:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761313549.034279:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f364ec74 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307454144:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761313549.280733:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313549.346825:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313550.304804:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313550.372877:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313551.328922:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313551.392898:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313552.352788:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313552.417729:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313553.376786:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313553.449985:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761313554.147259:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313554.147271:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761313554.147275:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761313554.147290:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1797 00000100:00080000:0.0:1761313554.147295:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761313554.147302:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761313554.147314:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761313554.147333:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b5a39135 x1846869307454400/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761313554.147353:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313554.147356:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761313554.147545:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307454208:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313554.147689:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307454336:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313554.147729:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761313554.147735:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307454272:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313554.148984:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307454400:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761313554.160357:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307454272:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313554.160720:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307454208:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313554.160738:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307454336:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761313554.160842:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313554.160848:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313554.160852:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307454400:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761313554.401536:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313554.465412:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313555.424966:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313555.489051:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313556.452843:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313556.513122:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313557.473316:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313557.538523:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313558.499893:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313558.560865:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761313559.265156:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313559.265166:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761313559.265170:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761313559.265184:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1802 00000100:00080000:2.0:1761313559.265188:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761313559.265194:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761313559.265206:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761313559.265224:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000e1be9a7 x1846869307454656/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761313559.265254:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313559.265258:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761313559.265295:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e662bceb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307454528:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313559.265299:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e1be9a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307454656:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761313559.266006:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002292ab57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307454464:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313559.266366:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b3aaddc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307454592:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761313559.271259:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761313559.271274:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761313559.271285:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e1be9a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307454656:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761313559.271908:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002292ab57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307454464:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313559.271926:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b3aaddc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307454592:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313559.272028:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e662bceb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307454528:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313559.521554:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313559.585190:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313560.544796:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313560.610680:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313561.568792:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313561.633421:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313562.598838:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313562.660716:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313563.619599:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313563.687114:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761313564.386310:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313564.386317:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761313564.386320:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761313564.386330:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1807 00000100:00080000:3.0:1761313564.386333:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761313564.386340:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761313564.386349:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761313564.386363:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000041c8b6cb x1846869307454912/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761313564.386385:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313564.386389:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761313564.386412:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000009911ad3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307454784:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313564.386526:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1761313564.386532:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307454720:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313564.386557:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e58cc722 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307454848:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313564.386881:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000041c8b6cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307454912:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761313564.391867:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e58cc722 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307454848:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313564.391885:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000009911ad3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307454784:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313564.406505:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307454720:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761313564.407208:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761313564.407213:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761313564.407218:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000041c8b6cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307454912:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761313564.641955:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313564.704854:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313565.665689:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313565.728942:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313566.688764:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313566.752952:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313567.713412:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313567.778352:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313568.736756:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313568.805742:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761313569.504819:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313569.504826:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761313569.504830:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761313569.504840:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1812 00000100:00080000:1.0:1761313569.504844:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761313569.504849:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761313569.504857:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761313569.504870:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a15ef099 x1846869307455168/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761313569.504888:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313569.504891:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761313569.504913:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307455040:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313569.505007:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1761313569.505014:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307454976:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313569.505038:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307455104:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313569.505222:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307455168:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00080000:3.0:1761313569.509943:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313569.509954:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313569.509963:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307455168:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761313569.511003:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307454976:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313569.511035:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307455040:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313569.511321:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307455104:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313569.760813:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313569.825188:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313570.784771:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313570.855077:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313571.808749:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313571.873682:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313572.833125:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313572.898917:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313573.856763:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313573.920848:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761313574.625117:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313574.625125:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761313574.625128:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761313574.625139:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1817 00000100:00080000:1.0:1761313574.625142:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761313574.625148:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761313574.625157:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761313574.625172:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a1719d14 x1846869307455424/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761313574.625195:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313574.625198:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761313574.625220:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307455296:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313574.625238:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307455424:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761313574.625306:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1761313574.625312:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307455232:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313574.625334:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307455360:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313574.637088:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307455296:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761313574.637144:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313574.637149:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313574.637153:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307455424:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761313574.637219:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307455360:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313574.639832:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307455232:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313574.880775:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313574.944850:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313575.904764:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313575.968828:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313576.928733:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313576.997085:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313577.958340:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313578.016843:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313578.976780:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313579.040845:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761313579.744829:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313579.744836:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761313579.744839:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761313579.744849:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1822 00000100:00080000:1.0:1761313579.744852:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761313579.744859:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761313579.744869:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761313579.744883:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000009cec3faa x1846869307455680/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761313579.744903:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313579.744906:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761313579.745006:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307455680:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761313579.745008:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307455488:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313579.745405:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307455616:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313579.745450:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761313579.745462:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307455552:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761313579.752533:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313579.752547:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313579.752558:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307455680:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761313579.752707:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307455552:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313579.752718:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307455488:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313579.752728:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307455616:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313580.000770:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313580.065629:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313581.024843:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313581.089286:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313582.048809:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313582.112813:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313583.073145:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313583.139089:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313584.096872:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313584.160844:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761313584.864823:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313584.864831:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761313584.864834:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761313584.864845:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1827 00000100:00080000:1.0:1761313584.864848:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00100000:0.0:1761313584.864856:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307455744:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761313584.864856:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761313584.864869:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761313584.864885:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000009cec3faa x1846869307455936/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761313584.864910:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313584.864913:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313584.864941:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307455872:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313584.864979:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761313584.864985:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307455808:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313584.865122:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307455936:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00080000:3.0:1761313584.871954:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313584.871965:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313584.871974:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307455936:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761313584.872931:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307455808:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313584.872966:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307455744:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313584.872977:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307455872:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313585.126507:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313585.185490:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313586.145887:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313586.210660:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313587.169242:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313587.234783:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313588.192754:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313588.256825:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313589.216792:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313589.280870:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313589.985723:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e662bceb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307456000:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313589.986661:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ee08055d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307456064:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313589.987285:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761313589.987293:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000011b5c99f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307456128:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761313589.988733:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313589.988743:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761313589.988746:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761313589.988755:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1832 00000100:00080000:2.0:1761313589.988758:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761313589.988763:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761313589.988774:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761313589.988801:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008b3aaddc x1846869307456192/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761313589.988832:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313589.988835:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313589.988895:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b3aaddc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307456192:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761313589.994188:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000011b5c99f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307456128:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313589.994210:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ee08055d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307456064:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761313589.994271:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761313589.994276:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761313589.994280:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b3aaddc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307456192:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761313589.995588:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e662bceb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307456000:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313590.240896:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313590.305263:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313591.265716:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313591.330547:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313592.289671:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313592.353367:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313593.312713:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313593.377424:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313594.336753:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313594.400862:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761313595.104840:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313595.104847:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761313595.104850:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761313595.104861:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1838 00000100:00080000:1.0:1761313595.104864:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761313595.104870:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761313595.104879:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761313595.104893:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a1719d14 x1846869307456448/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761313595.104921:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313595.104924:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313595.105012:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307456256:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313595.105095:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307456384:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313595.105127:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761313595.105133:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307456320:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313595.105181:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307456448:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761313595.117154:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307456256:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761313595.117592:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313595.117598:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313595.117603:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307456448:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761313595.117685:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307456320:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313595.117696:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307456384:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313595.361674:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313595.425565:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313596.386611:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313596.448982:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313597.408819:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313597.474078:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313598.433093:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313598.496840:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313599.458794:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313599.521293:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761313600.224873:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307456512:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313600.225071:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761313600.225078:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307456576:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761313600.226729:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313600.226736:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761313600.226739:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761313600.226748:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1843 00000100:00080000:1.0:1761313600.226751:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761313600.226757:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761313600.226767:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761313600.226781:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000163e3801 x1846869307456704/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761313600.226834:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307456704:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761313600.228397:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307456640:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761313600.231611:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313600.231653:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761313600.242715:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307456640:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313600.242731:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307456576:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313600.242741:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307456512:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761313600.242808:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313600.242820:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313600.242831:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307456704:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761313600.480800:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313600.547530:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313601.508386:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313601.570750:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313602.529234:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313602.595311:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313603.554135:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313603.616933:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313604.576797:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313604.644113:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761313605.344896:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313605.344906:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761313605.344911:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761313605.344925:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1848 00000100:00080000:1.0:1761313605.344929:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761313605.344937:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761313605.344951:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761313605.344971:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a15ef099 x1846869307456960/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761313605.344997:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313605.345002:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761313605.345038:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307456832:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313605.345153:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1761313605.345160:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307456768:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313605.345186:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307456896:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313605.345326:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307456960:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00080000:3.0:1761313605.350223:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313605.350237:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313605.350248:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307456960:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761313605.350358:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307456768:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313605.350398:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307456896:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313605.350410:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307456832:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313605.600780:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313605.665376:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313606.624759:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313606.689254:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313607.648765:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313607.712899:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313608.672767:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313608.742253:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313609.696785:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313609.762065:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761313610.465348:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313610.465356:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761313610.465359:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761313610.465369:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1853 00000100:00080000:2.0:1761313610.465372:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761313610.465379:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761313610.465388:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761313610.465405:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008a641e2e x1846869307457216/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761313610.465431:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313610.465434:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313610.465518:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008a641e2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307457216:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761313610.465550:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002292ab57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307457024:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313610.465756:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000015f165e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307457152:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313610.465799:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761313610.465806:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e1be9a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307457088:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313610.473025:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e1be9a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307457088:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313610.473043:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000015f165e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307457152:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313610.473768:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002292ab57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307457024:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761313610.473859:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761313610.473865:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761313610.473869:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008a641e2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307457216:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761313610.722353:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313610.784894:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313611.744823:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313611.810185:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313612.769563:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313612.832889:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313613.792788:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313613.856875:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313614.818703:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313614.880972:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761313615.586366:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313615.586377:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761313615.586380:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761313615.586394:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1858 00000100:00080000:1.0:1761313615.586399:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761313615.586406:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761313615.586420:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761313615.586440:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000eb8b1f19 x1846869307457472/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761313615.586464:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313615.586468:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761313615.586543:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307457472:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761313615.586983:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307457280:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313615.587019:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307457408:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313615.587042:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761313615.587047:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307457344:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761313615.594448:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761313615.594458:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761313615.594467:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307457472:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761313615.598391:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307457344:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313615.598410:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307457280:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313615.598420:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307457408:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313615.840759:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313615.904856:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313616.864918:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313616.936519:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313617.888765:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313617.953819:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313618.912779:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313618.979881:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313619.936788:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313620.000852:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761313620.704842:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307457536:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313620.705809:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307457600:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761313620.708759:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313620.708768:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761313620.708772:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761313620.708783:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1863 00000100:00080000:1.0:1761313620.708787:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761313620.708808:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761313620.708818:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761313620.708838:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ed2f9bca x1846869307457728/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761313620.708866:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313620.708870:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761313620.708900:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761313620.708913:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307457664:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313620.709342:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307457728:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761313620.718316:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307457600:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313620.720287:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307457536:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313620.722206:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307457664:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761313620.722287:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761313620.722292:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761313620.722296:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307457728:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761313620.961197:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313621.025578:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313621.984766:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313622.052065:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313623.008773:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313623.072834:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313624.033053:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313624.097633:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313625.056802:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313625.121031:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761313625.825382:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313625.825391:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761313625.825413:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761313625.825425:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1868 00000100:00080000:2.0:1761313625.825429:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761313625.825435:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761313625.825449:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761313625.825468:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b2298797 x1846869307457984/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761313625.825495:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313625.825498:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761313625.825635:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e3466f5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307457792:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761313625.825860:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ffb03e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307457856:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313625.825909:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eca34e1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307457920:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313625.825961:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b2298797 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307457984:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761313625.837305:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ffb03e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307457856:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313625.837408:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e3466f5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307457792:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761313625.837419:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:3.0:1761313625.837422:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eca34e1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307457920:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761313625.837425:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761313625.837429:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b2298797 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307457984:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761313626.080800:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313626.144876:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313627.104821:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313627.168856:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313628.128814:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313628.193466:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313629.152797:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313629.216837:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313630.176803:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313630.241209:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761313630.945262:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313630.945275:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761313630.945280:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761313630.945293:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1873 00000100:00080000:2.0:1761313630.945298:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761313630.945305:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761313630.945321:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761313630.945342:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000aea3302e x1846869307458240/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761313630.945430:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aea3302e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307458240:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761313630.945900:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ffb03e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307458048:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313630.945957:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ae09c97d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307458176:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313630.946078:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761313630.946087:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039978a4e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307458112:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313630.948272:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ffb03e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307458048:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761313630.949336:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761313630.949344:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761313630.949351:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aea3302e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307458240:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00080000:2.0:1761313630.949654:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313630.949669:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761313630.950300:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ae09c97d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307458176:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313630.956204:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039978a4e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307458112:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313631.202347:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313631.265588:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313632.224806:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313632.291727:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313633.248802:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313633.314647:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313634.272874:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313634.336911:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313635.297549:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313635.360991:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761313636.066988:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313636.066998:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761313636.067002:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761313636.067018:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1879 00000100:00080000:2.0:1761313636.067022:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761313636.067031:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761313636.067045:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761313636.067063:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ba5c8e77 x1846869307458496/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761313636.067085:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313636.067089:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761313636.067500:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e9376149 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307458304:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313636.067604:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004592c0a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307458432:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313636.067947:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761313636.068020:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001426be00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307458368:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313636.068349:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ba5c8e77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307458496:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00080000:1.0:1761313636.079698:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761313636.079721:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761313636.079732:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ba5c8e77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307458496:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761313636.079829:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001426be00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307458368:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313636.079842:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e9376149 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307458304:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313636.079851:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004592c0a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307458432:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313636.321502:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313636.384922:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313637.345148:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313637.408818:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313638.368809:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313638.434871:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313639.392838:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313639.457126:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313640.416813:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313640.481022:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313641.185042:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307458560:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761313641.187078:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1761313641.187135:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307458688:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313641.187206:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761313641.187212:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307458624:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761313641.187363:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761313641.187368:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761313641.187373:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1884 00000100:00080000:0.0:1761313641.187375:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761313641.187380:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761313641.187390:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761313641.187406:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000005b900d12 x1846869307458752/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761313641.187429:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313641.187433:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761313641.187496:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307458752:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761313641.189687:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307458560:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313641.194405:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307458688:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313641.196268:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307458624:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761313641.196442:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761313641.196446:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761313641.196451:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307458752:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761313641.440797:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313641.504866:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313642.465920:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313642.532786:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313643.490060:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313643.555029:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313644.512757:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313644.576877:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313645.536782:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313645.600970:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761313646.317875:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313646.317882:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761313646.317885:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761313646.317895:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1889 00000100:00080000:1.0:1761313646.317907:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761313646.317913:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761313646.317921:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761313646.317934:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000eb8b1f19 x1846869307459008/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761313646.317973:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313646.317976:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761313646.318007:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307458880:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313646.318094:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1761313646.318100:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307458816:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313646.318124:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307458944:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313646.318420:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307459008:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761313646.331218:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307458816:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313646.331247:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307458944:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313646.331258:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307458880:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761313646.333309:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761313646.333323:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761313646.333336:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307459008:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761313646.565743:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313646.625378:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313647.584809:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313647.649623:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313648.608818:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313648.672832:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313649.632782:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313649.697087:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313650.658522:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313650.721093:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761313651.424857:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313651.424874:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761313651.424878:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761313651.424891:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1894 00000100:00080000:1.0:1761313651.424896:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761313651.424903:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761313651.424916:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761313651.424933:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a15ef099 x1846869307459264/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761313651.424957:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313651.424961:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761313651.425019:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307459072:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313651.425122:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307459200:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313651.425158:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761313651.425163:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307459136:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313651.425692:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307459264:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00080000:2.0:1761313651.432291:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761313651.432306:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761313651.432319:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307459264:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761313651.432835:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307459200:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313651.432860:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307459136:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313651.432884:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307459072:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313651.682841:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313651.746187:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313652.704819:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313652.771263:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313653.729563:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313653.793035:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313654.754516:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313654.816902:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313655.776777:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313655.840908:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313656.544968:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307459328:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761313656.544991:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313656.544998:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761313656.545002:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761313656.545013:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1899 00000100:00080000:0.0:1761313656.545017:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761313656.545024:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761313656.545034:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761313656.545052:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000034a6cd79 x1846869307459520/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761313656.545100:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307459456:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313656.545133:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761313656.545138:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307459520:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761313656.545145:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307459392:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761313656.546766:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313656.546775:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313656.554297:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307459328:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761313656.560875:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761313656.560888:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761313656.560900:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307459520:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761313656.561558:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307459392:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313656.561575:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307459456:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313656.801208:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313656.867396:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313657.825213:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313657.888886:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313658.848852:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313658.913253:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313659.872898:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313659.936943:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313660.896869:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313660.963863:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761313661.667737:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307459584:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313661.670681:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761313661.670707:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307459648:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761313661.670978:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313661.670984:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761313661.670988:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761313661.670994:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1904 00000100:00080000:1.0:1761313661.670997:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761313661.671004:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761313661.671016:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761313661.671017:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761313661.671026:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307459712:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313661.671034:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000540c3ce6 x1846869307459776/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761313661.671062:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313661.671066:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761313661.671179:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307459776:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761313661.677714:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307459712:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313661.677853:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307459648:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761313661.677968:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761313661.677974:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761313661.677979:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307459776:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761313661.680868:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307459584:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313661.920839:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313661.984952:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313662.944882:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313663.009127:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313663.968807:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313664.033285:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313664.992916:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313665.057185:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313666.016823:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313666.082108:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761313666.785014:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313666.785024:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761313666.785028:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761313666.785041:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1909 00000100:00080000:3.0:1761313666.785045:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761313666.785051:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761313666.785063:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761313666.785081:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000bc14d8e x1846869307460032/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761313666.785129:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313666.785133:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761313666.785165:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000009911ad3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307459904:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313666.785296:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1761313666.785303:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307459840:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313666.785331:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e58cc722 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307459968:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313666.787332:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307460032:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761313666.789218:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307459840:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313666.792315:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000009911ad3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307459904:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313666.792447:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e58cc722 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307459968:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761313666.793617:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761313666.793624:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761313666.793631:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307460032:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761313667.040813:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313667.105562:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313668.064785:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313668.128927:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313669.088803:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313669.154536:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313670.114513:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313670.176925:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313671.136948:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313671.201711:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761313671.904838:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313671.904847:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761313671.904851:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1761313671.904977:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307460096:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313671.905096:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307460224:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313671.905133:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761313671.905139:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307460160:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761313671.905535:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1914 00000100:00080000:1.0:1761313671.905545:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761313671.905553:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761313671.905566:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761313671.905593:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000eb8b1f19 x1846869307460288/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761313671.905726:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313671.905730:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761313671.906741:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307460288:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761313671.908514:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307460096:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313671.909472:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307460160:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313671.909493:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307460224:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761313671.918828:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761313671.918842:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761313671.918854:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307460288:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761313672.162829:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313672.225030:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313673.184806:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313673.248959:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313674.208767:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313674.275193:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313675.232792:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313675.297727:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313676.257164:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313676.322936:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761313677.024872:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313677.024880:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761313677.024883:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761313677.024893:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1919 00000100:00080000:0.0:1761313677.024896:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761313677.024902:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761313677.024911:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761313677.024926:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000094820351 x1846869307460544/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761313677.024948:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313677.024951:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761313677.024982:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307460544:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761313677.025004:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307460352:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313677.025062:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307460480:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313677.025089:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761313677.025094:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307460416:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:02020000:2.0:1761313677.035119:0:2262:0:(client.c:1361:ptlrpc_check_status()) 11-0: lustre-OST0000-osc-ffff951bc4d8c800: operation ost_connect to node 192.168.203.151@tcp failed: rc = -19 00000100:00100000:0.0:1761313677.035606:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307460352:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313677.035623:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307460480:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313677.041559:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307460416:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761313677.042716:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761313677.042723:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761313677.042731:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307460544:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761313677.280973:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313677.347757:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313678.304799:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313678.369546:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313679.328797:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313679.392931:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313680.352819:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313680.418476:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313681.377777:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313681.440873:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761313682.145125:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313682.145133:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761313682.145137:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761313682.145149:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1925 00000100:00080000:0.0:1761313682.145152:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761313682.145160:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000100:00100000:1.0:1761313682.145161:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307460608:192.168.203.151@tcp:400:kworker/u8:3.0 00000008:00100000:0.0:1761313682.145170:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761313682.145192:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000084f3d917 x1846869307460800/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761313682.145260:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307460736:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761313682.145283:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307460800:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761313682.145294:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761313682.145302:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307460672:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761313682.145385:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313682.145390:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761313682.148413:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307460608:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313682.148464:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307460672:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313682.149457:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307460736:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761313682.149457:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761313682.149462:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761313682.149467:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307460800:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761313682.400739:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313682.465501:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313683.431872:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313683.490348:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313684.450125:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313684.514622:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313685.472760:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313685.536975:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313686.496769:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313686.561148:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313687.264878:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307460864:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313687.268299:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307460864:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313687.269456:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307460928:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761313687.271665:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313687.271685:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761313687.271688:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761313687.271697:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1930 00000100:00080000:0.0:1761313687.271701:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761313687.271706:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761313687.271716:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761313687.271731:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007a68d9a8 x1846869307461056/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761313687.271755:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313687.271758:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313687.271784:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761313687.271798:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307460992:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761313687.272631:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307461056:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761313687.275711:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307460992:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313687.275727:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307460928:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761313687.275969:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761313687.275974:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761313687.275978:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307461056:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761313687.520777:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313687.584879:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313688.544774:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313688.610212:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313689.568765:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313689.633239:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313690.594031:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313690.657085:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313691.616768:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313691.681058:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761313692.385101:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313692.385108:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761313692.385111:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761313692.385120:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1935 00000100:00080000:1.0:1761313692.385123:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761313692.385129:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761313692.385137:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761313692.385167:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000003871442b x1846869307461312/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761313692.385194:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313692.385197:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761313692.385222:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307461184:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313692.385307:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1761313692.385313:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307461312:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761313692.385316:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307461120:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313692.385349:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307461248:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761313692.389277:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761313692.389288:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761313692.389298:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307461312:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761313692.390310:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307461120:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313692.390328:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307461248:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313692.390338:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307461184:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313692.643334:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313692.706100:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313693.665608:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313693.728926:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313694.688942:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313694.752996:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313695.712817:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761313695.782671:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313696.737041:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313696.800914:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313697.505368:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000009911ad3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307461376:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761313697.511725:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313697.511735:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761313697.511739:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761313697.511752:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1940 00000100:00080000:3.0:1761313697.511756:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761313697.511763:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000100:00100000:2.0:1761313697.511885:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307461440:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313697.512444:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761313697.512455:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d32bed84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307461504:192.168.203.151@tcp:400:kworker/u8:0.0 00000008:00100000:3.0:1761313697.514821:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761313697.514884:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000bc14d8e x1846869307461568/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761313697.514922:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313697.514927:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761313697.515335:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307461568:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761313697.515711:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000009911ad3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307461376:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313697.518548:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d32bed84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307461504:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313697.518570:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307461440:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761313697.518676:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761313697.518681:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761313697.518686:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307461568:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761313697.760907:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313697.825312:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313698.784889:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761313698.849044:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313699.808807:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313699.872923:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313700.834059:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313700.897845:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313701.856758:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761313701.920876:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761313702.625152:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313702.625159:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761313702.625162:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761313702.625172:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1945 00000100:00080000:0.0:1761313702.625174:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761313702.625180:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761313702.625189:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761313702.625204:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007a68d9a8 x1846869307461824/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761313702.625228:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313702.625243:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313702.625264:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307461696:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313702.625339:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1761313702.625344:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307461632:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313702.625360:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307461760:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761313702.625660:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307461824:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00080000:2.0:1761313702.628589:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761313702.628599:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761313702.628608:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307461824:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761313702.628781:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307461632:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313702.634379:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307461696:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313702.635744:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307461760:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313702.882484:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313702.945108:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313703.904824:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313703.969819:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313704.928760:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761313704.995947:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313705.952852:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313706.016897:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313706.976772:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313707.041588:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761313707.744863:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313707.744870:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761313707.744873:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761313707.744884:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1950 00000100:00080000:0.0:1761313707.744886:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761313707.744892:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761313707.744902:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761313707.744916:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000084f3d917 x1846869307462080/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761313707.744939:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313707.744943:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761313707.745233:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307462080:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761313707.746688:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307461888:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313707.746764:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307462016:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313707.746808:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307461952:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761313707.750821:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761313707.750832:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761313707.750841:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307462080:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761313707.757864:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307461952:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313707.757913:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307461888:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313707.757924:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307462016:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313708.000764:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761313708.065281:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313709.024763:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313709.088852:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313710.048771:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313710.113534:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313711.072741:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761313711.136803:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313712.097496:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313712.160892:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761313712.865020:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313712.865028:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761313712.865031:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761313712.865042:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1955 00000100:00080000:2.0:1761313712.865045:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761313712.865051:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761313712.865060:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761313712.865074:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000039eebc2 x1846869307462336/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761313712.865100:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313712.865119:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761313712.865149:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b4eac89a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307462208:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313712.865150:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000039eebc2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307462336:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761313712.865231:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000007136630 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307462144:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313712.865510:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307462272:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761313712.873146:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761313712.873161:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761313712.873173:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000039eebc2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307462336:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761313712.874088:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307462272:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313712.874372:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b4eac89a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307462208:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313712.874956:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000007136630 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307462144:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313713.123094:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313713.186243:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313714.146513:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761313714.208826:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313715.168797:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313715.232888:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313716.192789:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313716.257298:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313717.216810:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761313717.280787:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761313717.985297:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313717.985304:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761313717.985307:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761313717.985316:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1960 00000100:00080000:0.0:1761313717.985320:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761313717.985325:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761313717.985332:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761313717.985346:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007dcceb4a x1846869307462592/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761313717.985367:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313717.985371:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313717.985422:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307462464:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313717.985503:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1761313717.985509:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307462400:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313717.985534:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307462528:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761313717.985801:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307462592:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761313717.991121:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307462528:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313717.991140:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307462464:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313717.997601:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307462400:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761313718.000499:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761313718.000510:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761313718.000518:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307462592:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761313718.241079:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313718.305066:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313719.264772:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313719.328856:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313720.289184:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761313720.354453:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313721.312769:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313721.376839:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313722.336888:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313722.404938:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761313723.105034:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313723.105042:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761313723.105045:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761313723.105054:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1966 00000100:00080000:0.0:1761313723.105069:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761313723.105074:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761313723.105083:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761313723.105099:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008e59d025 x1846869307462848/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761313723.105166:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307462848:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761313723.107727:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307462656:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313723.107800:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307462784:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313723.107825:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761313723.107830:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307462720:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761313723.109409:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313723.109417:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00080000:2.0:1761313723.114339:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761313723.114363:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761313723.114376:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307462848:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761313723.114480:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307462656:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313723.114506:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307462784:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313723.114778:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307462720:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313723.360822:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761313723.424948:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313724.384759:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313724.448910:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313725.410687:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313725.472882:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313726.433579:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761313726.499529:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313727.456800:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313727.520866:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313728.226255:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b4eac89a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307462912:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313728.227838:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000039eebc2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307462976:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761313728.228715:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313728.228722:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761313728.228745:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761313728.228754:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1971 00000100:00080000:2.0:1761313728.228758:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761313728.228765:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761313728.228777:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761313728.228799:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000007136630 x1846869307463104/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761313728.228827:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313728.228831:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761313728.228862:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761313728.228871:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f220504f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307463040:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313728.229476:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000007136630 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307463104:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761313728.234478:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b4eac89a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307462912:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313728.234604:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f220504f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307463040:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313728.234618:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000039eebc2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307462976:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761313728.236585:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761313728.236596:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761313728.236607:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000007136630 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307463104:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761313728.480921:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313728.546696:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313729.504834:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:2.0:1761313729.568824:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313730.533666:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313730.593007:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313731.552967:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313731.619730:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313732.576820:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313732.641464:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761313733.344983:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313733.344990:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761313733.344994:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761313733.345004:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1976 00000100:00080000:0.0:1761313733.345007:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761313733.345013:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761313733.345024:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761313733.345042:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000094820351 x1846869307463360/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761313733.345195:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307463168:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761313733.345304:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313733.345308:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313733.345345:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307463232:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761313733.345749:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307463360:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761313733.345843:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307463296:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313733.349780:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307463168:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761313733.354736:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761313733.354745:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761313733.354756:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307463360:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761313733.355551:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307463296:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313733.355599:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307463232:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313733.600757:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313733.665855:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313734.624764:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313734.688863:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313735.651467:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313735.714744:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313736.672815:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313736.739792:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313737.696795:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313737.762015:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761313738.467356:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313738.467363:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761313738.467366:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761313738.467376:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1981 00000100:00080000:2.0:1761313738.467392:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761313738.467398:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761313738.467406:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761313738.467422:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000004592c0a5 x1846869307463616/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761313738.467446:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313738.467449:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761313738.467522:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004592c0a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307463616:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761313738.469212:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000039eebc2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307463424:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313738.469310:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307463552:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313738.469339:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761313738.469344:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f220504f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307463488:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761313738.476480:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761313738.476491:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761313738.476501:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004592c0a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307463616:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761313738.477180:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f220504f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307463488:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313738.477197:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000039eebc2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307463424:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313738.477207:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307463552:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313738.721705:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313738.785799:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313739.745131:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313739.809484:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313740.768788:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313740.834150:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313741.793438:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313741.857386:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313742.816811:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313742.881109:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761313743.584947:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313743.584954:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761313743.584957:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761313743.584966:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1986 00000100:00080000:2.0:1761313743.584969:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761313743.584975:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761313743.584984:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761313743.584997:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000039978a4e x1846869307463872/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761313743.585025:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313743.585029:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313743.585245:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039978a4e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307463872:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761313743.586022:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e9376149 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307463680:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313743.586081:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ba5c8e77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307463808:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313743.586108:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761313743.586114:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001426be00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307463744:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313743.593898:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001426be00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307463744:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313743.593935:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ba5c8e77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307463808:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313743.599377:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e9376149 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307463680:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761313743.599593:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761313743.599599:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761313743.599604:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039978a4e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307463872:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761313743.841257:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313743.905081:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313744.864834:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313744.929045:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313745.888768:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313745.952817:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313746.913996:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313746.976834:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313747.937305:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313748.000833:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761313748.704870:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313748.704879:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761313748.704882:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761313748.704893:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1991 00000100:00080000:0.0:1761313748.704896:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761313748.704903:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761313748.704912:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761313748.704930:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000546b1bfe x1846869307464128/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761313748.704947:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307463936:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761313748.704957:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313748.704962:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761313748.704991:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307464128:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761313748.705004:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307464000:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313748.705034:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307464064:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313748.712493:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307463936:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313748.712515:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307464064:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313748.712574:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307464000:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761313748.715540:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761313748.715548:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761313748.715557:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307464128:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761313748.960903:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313749.025264:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313749.985512:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313750.049707:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313751.008811:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313751.073183:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313752.032764:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313752.096808:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313753.058001:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313753.120905:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761313753.825686:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313753.825696:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761313753.825700:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761313753.825714:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 1996 00000100:00080000:3.0:1761313753.825718:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761313753.825726:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761313753.825739:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761313753.825756:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000cc6dd8ca x1846869307464384/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761313753.825789:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313753.825793:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761313753.825831:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000041c8b6cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307464256:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313753.825939:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1761313753.825945:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e58cc722 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307464192:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313753.825970:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f364ec74 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307464320:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313753.826376:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cc6dd8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307464384:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761313753.833748:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e58cc722 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307464192:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313753.833775:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f364ec74 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307464320:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313753.833787:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000041c8b6cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307464256:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761313753.834614:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761313753.834622:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761313753.834631:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc6dd8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307464384:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761313754.083473:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313754.145561:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313755.104831:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313755.169049:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313756.128827:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313756.194258:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313757.153142:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313757.217882:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313758.176891:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313758.241052:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313758.945572:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ae09c97d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307464448:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313758.945825:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761313758.945833:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aea3302e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307464512:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761313758.945958:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313758.945965:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761313758.945971:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761313758.945978:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2001 00000100:00080000:2.0:1761313758.945981:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761313758.945987:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761313758.946000:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761313758.946018:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b2298797 x1846869307464640/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761313758.946038:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313758.946042:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761313758.946070:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b2298797 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307464640:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761313758.946275:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761313758.946306:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ffb03e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307464576:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313758.953383:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ffb03e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307464576:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313758.953433:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aea3302e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307464512:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313758.953449:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ae09c97d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307464448:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761313758.955074:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761313758.955345:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761313758.955355:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b2298797 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307464640:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761313759.201235:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313759.265149:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313760.225577:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313760.289415:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313761.248866:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313761.313532:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313762.274418:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313762.336883:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313763.297926:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313763.360883:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761313764.064875:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313764.064882:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761313764.064886:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761313764.064896:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2007 00000100:00080000:3.0:1761313764.064899:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761313764.064905:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761313764.064916:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761313764.064934:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007673f690 x1846869307464896/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761313764.064963:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313764.064967:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761313764.064993:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f364ec74 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307464768:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313764.064999:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007673f690 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307464896:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761313764.065078:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000041c8b6cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307464704:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313764.067327:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e58cc722 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307464832:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313764.069168:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f364ec74 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307464768:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761313764.072345:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761313764.072371:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761313764.072382:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007673f690 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307464896:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761313764.072589:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000041c8b6cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307464704:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313764.072603:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e58cc722 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307464832:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313764.322221:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313764.388848:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313765.345187:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313765.411256:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313766.368762:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313766.432872:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313767.392835:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313767.457215:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313768.416800:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313768.480831:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761313769.184924:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313769.184933:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761313769.184936:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761313769.184947:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2012 00000100:00080000:1.0:1761313769.184950:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761313769.184956:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761313769.184966:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761313769.184981:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000003871442b x1846869307465152/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761313769.185012:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313769.185015:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761313769.185068:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307465152:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761313769.185071:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307465024:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313769.185407:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307464960:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313769.185508:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307465088:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761313769.191000:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761313769.191014:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761313769.191036:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307465152:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761313769.191885:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307465024:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313769.192065:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307464960:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313769.192079:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307465088:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313769.440759:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313769.504858:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313770.464807:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313770.529716:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313771.488768:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313771.552821:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313772.513276:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313772.576896:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313773.536760:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313773.600894:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761313774.304870:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307465216:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313774.305384:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761313774.305392:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307465280:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761313774.306874:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313774.306881:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761313774.306884:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761313774.306893:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2017 00000100:00080000:1.0:1761313774.306896:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761313774.306915:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761313774.306925:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761313774.306939:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a1719d14 x1846869307465408/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761313774.307033:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307465408:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00080000:1.0:1761313774.308814:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313774.308821:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761313774.308840:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761313774.308855:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307465344:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313774.321347:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307465280:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313774.321363:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307465216:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761313774.325857:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761313774.325868:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761313774.325878:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307465408:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761313774.325973:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307465344:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313774.561391:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313774.630427:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313775.584864:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313775.650307:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313776.608889:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313776.672854:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313777.632830:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313777.700414:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313778.657177:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313778.723483:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761313779.426335:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313779.426341:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761313779.426344:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761313779.426354:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2022 00000100:00080000:0.0:1761313779.426357:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761313779.426363:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761313779.426372:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761313779.426384:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000005b900d12 x1846869307465664/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761313779.426410:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313779.426424:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761313779.426466:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307465664:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761313779.426516:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307465472:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313779.426747:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307465600:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313779.427399:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761313779.427408:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307465536:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761313779.439126:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313779.439136:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313779.439145:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307465664:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761313779.439245:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307465536:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313779.439255:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307465472:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313779.439285:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307465600:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313779.680775:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313779.745134:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313780.704802:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313780.768860:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313781.729287:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313781.792861:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313782.752764:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313782.816921:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313783.776804:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313783.840895:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761313784.544827:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313784.544834:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761313784.544837:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761313784.544847:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2027 00000100:00080000:0.0:1761313784.544850:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761313784.544855:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761313784.544868:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761313784.544881:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000094820351 x1846869307465920/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761313784.544904:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313784.544907:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761313784.544960:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307465920:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761313784.545010:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307465728:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313784.545045:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307465856:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313784.545073:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761313784.545078:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307465792:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761313784.549728:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313784.549739:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313784.549747:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307465920:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761313784.549896:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307465792:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313784.549908:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307465856:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313784.553609:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307465728:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313784.800802:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313784.864866:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313785.825305:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313785.889784:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313786.848799:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313786.912989:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313787.872822:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313787.936837:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313788.896795:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313788.960763:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761313789.665586:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313789.665594:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761313789.665597:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761313789.665606:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2032 00000100:00080000:0.0:1761313789.665610:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761313789.665615:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761313789.665625:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761313789.665655:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000fe4020c4 x1846869307466176/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761313789.665681:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313789.665684:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761313789.665733:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307466176:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761313789.666266:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307465984:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313789.666317:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307466112:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313789.666347:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761313789.666355:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307466048:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761313789.671603:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313789.671612:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313789.671621:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307466176:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761313789.671962:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307465984:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313789.674477:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307466048:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313789.676058:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307466112:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313789.921033:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313789.984860:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313790.944775:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313791.010559:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313791.968767:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313792.033235:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313792.992759:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313793.056881:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313794.017140:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313794.080834:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761313794.790110:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313794.790117:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761313794.790120:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761313794.790130:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2037 00000100:00080000:1.0:1761313794.790133:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761313794.790139:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761313794.790147:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761313794.790161:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a1719d14 x1846869307466432/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761313794.790185:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313794.790188:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761313794.790208:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307466304:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313794.790284:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1761313794.790289:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307466240:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313794.790311:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307466368:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313794.790529:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307466432:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00080000:3.0:1761313794.793469:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313794.793479:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313794.793499:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307466432:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761313794.793820:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307466240:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313794.794283:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307466304:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313794.794301:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307466368:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313795.040967:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313795.106549:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313796.065488:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313796.141042:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313797.089490:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313797.153143:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313798.114952:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313798.177263:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313799.136742:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313799.201326:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761313799.905509:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313799.905517:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761313799.905520:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1761313799.905535:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b2298797 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307466496:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761313799.905695:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2042 00000100:00080000:2.0:1761313799.905699:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761313799.905704:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761313799.905714:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761313799.905731:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006e3466f5 x1846869307466688/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761313799.905748:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313799.905751:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761313799.905767:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ffb03e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307466560:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313799.907487:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e3466f5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307466688:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761313799.907490:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eca34e1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307466624:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313799.911487:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b2298797 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307466496:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313799.911515:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eca34e1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307466624:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313799.911624:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ffb03e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307466560:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761313799.912417:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761313799.912428:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761313799.912437:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e3466f5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307466688:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761313800.160824:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313800.225332:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313801.184981:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313801.251063:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313802.208759:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313802.273975:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313803.233266:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313803.296858:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313804.257234:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313804.321333:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761313805.030900:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313805.030907:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761313805.030911:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761313805.030922:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2047 00000100:00080000:1.0:1761313805.030926:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761313805.030933:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761313805.030943:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761313805.030969:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307466816:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313805.031068:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a15ef099 x1846869307466944/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761313805.031090:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00080000:1.0:1761313805.031098:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1761313805.031101:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307466752:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761313805.031109:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313805.031138:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307466880:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761313805.031287:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307466944:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761313805.039563:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307466816:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313805.042530:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307466752:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761313805.048186:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761313805.048194:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761313805.048203:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307466944:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761313805.048287:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307466880:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313805.280757:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313805.346480:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313806.305614:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313806.369059:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313807.328809:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313807.392874:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313808.352829:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313808.418018:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313809.376847:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313809.440950:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761313810.146060:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313810.146070:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761313810.146074:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761313810.146089:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2053 00000100:00080000:2.0:1761313810.146094:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761313810.146103:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761313810.146117:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761313810.146139:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000015f165e6 x1846869307467200/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761313810.146170:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313810.146174:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761313810.146206:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ffb03e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307467008:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313810.146270:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000015f165e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307467200:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761313810.146335:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002292ab57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307467136:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313810.146390:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761313810.146399:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008a641e2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307467072:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313810.154172:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008a641e2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307467072:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313810.154195:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ffb03e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307467008:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313810.154206:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002292ab57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307467136:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761313810.154323:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761313810.154329:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761313810.154333:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000015f165e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307467200:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761313810.400808:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313810.464965:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313811.424827:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313811.489241:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313812.448819:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313812.512939:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313813.476206:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313813.536922:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313814.497919:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313814.562005:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761313815.265989:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313815.265999:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761313815.266003:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1761313815.266013:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307467264:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761313815.266014:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2058 00000100:00080000:0.0:1761313815.266019:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761313815.266028:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761313815.266042:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761313815.266061:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000028de4b9d x1846869307467456/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761313815.266097:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313815.266123:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761313815.266130:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307467392:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313815.266155:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307467328:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313815.266355:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307467456:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761313815.272180:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307467328:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313815.272302:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307467264:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313815.272315:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307467392:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761313815.275239:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761313815.275255:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761313815.275266:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307467456:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761313815.521528:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313815.586297:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313816.547056:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313816.608902:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313817.568746:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313817.633044:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313818.595094:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313818.658454:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313819.616817:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313819.680942:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761313820.384893:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313820.384900:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761313820.384904:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761313820.384915:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2063 00000100:00080000:0.0:1761313820.384918:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00100000:1.0:1761313820.384923:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307467520:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761313820.384924:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761313820.384933:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761313820.384948:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000028de4b9d x1846869307467712/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761313820.384971:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313820.384974:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313820.384997:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307467584:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313820.384997:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307467648:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313820.385070:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307467712:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761313820.392881:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307467520:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313820.396457:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307467648:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313820.401708:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307467584:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761313820.402310:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313820.402316:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313820.402322:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307467712:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761313820.641011:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313820.709494:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313821.670100:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313821.728889:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313822.693538:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313822.752834:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313823.713788:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313823.778471:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313824.736870:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313824.800867:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761313825.504902:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313825.504909:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761313825.504912:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761313825.504923:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2068 00000100:00080000:0.0:1761313825.504926:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761313825.504932:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761313825.504941:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761313825.504957:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ace4d025 x1846869307467968/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761313825.504983:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313825.504986:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313825.505018:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307467840:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313825.505101:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1761313825.505107:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307467776:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313825.505131:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307467904:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313825.507552:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307467968:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761313825.514375:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307467776:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313825.514396:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307467904:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313825.514408:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307467840:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761313825.520298:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313825.520308:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313825.520318:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307467968:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761313825.760773:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313825.825587:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313826.784789:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313826.848920:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313827.808814:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313827.872919:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313828.832999:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313828.902453:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313829.856738:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313829.920861:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761313830.624904:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313830.624923:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761313830.624926:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761313830.624936:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2073 00000100:00080000:1.0:1761313830.624940:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761313830.624946:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761313830.624955:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761313830.624971:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000003871442b x1846869307468224/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761313830.624999:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313830.625002:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761313830.625028:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307468096:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313830.625108:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1761313830.625114:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307468032:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313830.625137:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307468160:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313830.625419:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307468224:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761313830.629104:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307468096:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313830.633776:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307468160:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313830.633948:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307468032:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:02020000:3.0:1761313830.634022:0:2262:0:(client.c:1361:ptlrpc_check_status()) 11-0: lustre-OST0000-osc-ffff951bc4d8c800: operation ost_connect to node 192.168.203.151@tcp failed: rc = -19 00000100:00080000:3.0:1761313830.641956:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313830.641966:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313830.641978:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307468224:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761313830.880854:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313830.944909:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313831.904769:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313831.968979:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313832.929233:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313832.998082:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313833.952752:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313834.016827:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313834.976772:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313835.043712:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761313835.745052:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313835.745059:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761313835.745062:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761313835.745073:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2078 00000100:00080000:2.0:1761313835.745077:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 6s 00000100:00080000:2.0:1761313835.745098:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761313835.745108:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761313835.745123:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000e1be9a7 x1846869307468480/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761313835.745210:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e1be9a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307468480:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761313835.747053:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002292ab57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307468288:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761313835.747222:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313835.747228:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761313835.747247:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ffb03e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307468352:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313835.748092:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008a641e2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307468416:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313835.753288:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002292ab57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307468288:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761313835.753332:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:3.0:1761313835.753333:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008a641e2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307468416:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761313835.753340:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761313835.753348:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e1be9a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307468480:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761313835.753524:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ffb03e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307468352:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313836.001416:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313836.064854:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313837.024775:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313837.088858:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313838.048763:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313838.114519:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313839.072769:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313839.136970:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313840.097050:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313840.161265:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761313840.866599:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313840.866606:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761313840.866609:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761313840.866619:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2083 00000100:00080000:3.0:1761313840.866623:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 11s 00000100:00080000:3.0:1761313840.866628:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761313840.866669:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761313840.866687:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000004c6aebc5 x1846869307468736/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761313840.866701:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313840.866704:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761313840.866729:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307468608:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313840.866841:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1761313840.866847:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f364ec74 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307468544:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313840.866872:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307468672:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313840.866968:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307468736:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00080000:0.0:1761313840.881559:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761313840.881569:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761313840.881579:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307468736:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761313840.882557:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f364ec74 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307468544:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313840.882573:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307468672:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313840.882583:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307468608:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761313841.120765:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313841.188082:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313842.144784:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313842.209153:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313843.171313:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313843.233626:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313844.192794:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313844.258824:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313845.218204:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313845.281176:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761313845.986252:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313845.986260:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761313845.986263:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761313845.986273:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2088 00000100:00080000:2.0:1761313845.986286:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 16s 00000100:00080000:2.0:1761313845.986292:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761313845.986301:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761313845.986316:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ee08055d x1846869307468992/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761313845.986344:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313845.986348:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761313845.986374:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e662bceb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307468864:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313845.986467:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1761313845.986473:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ffb03e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307468800:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313845.986510:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b3aaddc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307468928:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313845.986670:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ee08055d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307468992:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761313845.992608:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ffb03e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307468800:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313845.992636:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b3aaddc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307468928:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313845.992673:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e662bceb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307468864:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761313845.992772:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761313845.992778:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761313845.992782:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ee08055d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307468992:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761313846.241260:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313846.304811:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313847.264911:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313847.331292:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313848.288786:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313848.353145:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313849.312761:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313849.376910:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313850.337603:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313850.401118:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761313851.105730:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313851.105737:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761313851.105741:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1761313851.105744:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307469056:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761313851.105750:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2094 00000100:00080000:0.0:1761313851.105755:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761313851.105761:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761313851.105772:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761313851.105805:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007dcceb4a x1846869307469248/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761313851.105829:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307469184:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761313851.105834:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313851.105838:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313851.105863:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307469120:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313851.107095:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307469248:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761313851.110590:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307469184:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313851.112358:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307469120:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313851.112483:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307469056:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761313851.112594:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313851.112600:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313851.112604:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307469248:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761313851.361237:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313851.425209:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313852.384713:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313852.448846:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313853.408764:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313853.472886:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313854.432762:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313854.496880:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313855.458555:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313855.524879:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761313856.225587:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313856.225595:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761313856.225599:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761313856.225611:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2099 00000100:00080000:0.0:1761313856.225615:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761313856.225622:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761313856.225632:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761313856.225684:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000057b18867 x1846869307469504/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761313856.225697:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307469312:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313856.225772:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307469504:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761313856.225842:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307469440:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313856.225892:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761313856.225899:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307469376:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761313856.226998:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313856.227004:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313856.228587:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307469376:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313856.228608:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307469440:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313856.229342:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307469312:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761313856.233538:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313856.233601:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313856.233615:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307469504:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761313856.481567:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313856.544913:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313857.505370:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313857.568892:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313858.529556:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313858.594759:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313859.552807:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313859.617740:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313860.577529:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313860.645714:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761313861.344987:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313861.344996:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761313861.345000:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761313861.345013:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2104 00000100:00080000:2.0:1761313861.345016:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761313861.345023:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761313861.345034:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761313861.345051:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000011b5c99f x1846869307469760/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761313861.345076:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313861.345080:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761313861.345126:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b3aaddc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307469632:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313861.345584:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000011b5c99f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307469760:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761313861.345677:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1761313861.345684:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e662bceb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307469568:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313861.347728:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ffb03e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307469696:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761313861.357440:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761313861.357454:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761313861.357467:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000011b5c99f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307469760:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761313861.359235:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ffb03e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307469696:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313861.359253:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b3aaddc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307469632:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313861.360016:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e662bceb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307469568:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313861.600763:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313861.665705:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313862.624762:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313862.690512:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313863.648808:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313863.712893:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313864.672770:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313864.741230:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313865.697833:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313865.761964:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761313866.470208:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313866.470220:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761313866.470224:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761313866.470237:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2109 00000100:00100000:0.0:1761313866.470238:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307469888:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761313866.470241:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1761313866.470270:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761313866.470284:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761313866.470303:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000009cec3faa x1846869307470016/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761313866.470332:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761313866.470336:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313866.470371:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1761313866.470377:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307469824:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313866.470404:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307469952:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313866.474957:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307470016:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761313866.475719:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307469824:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313866.475739:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307469952:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313866.475752:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307469888:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761313866.479271:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313866.479285:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313866.479296:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307470016:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761313866.721268:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313866.793282:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313867.745138:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313867.809976:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313868.768805:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313868.832839:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313869.792768:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313869.857021:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313870.820490:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313870.880944:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761313871.586689:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313871.586697:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761313871.586700:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761313871.586712:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2114 00000100:00080000:0.0:1761313871.586715:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761313871.586720:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761313871.586730:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761313871.586746:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000fe4020c4 x1846869307470272/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761313871.586790:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313871.586794:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313871.586827:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307470080:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313871.586908:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307470208:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313871.586939:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761313871.586944:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307470144:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313871.587717:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307470272:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00080000:3.0:1761313871.590087:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313871.590096:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313871.590106:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307470272:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761313871.593902:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307470144:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313871.593918:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307470080:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313871.593927:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307470208:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313871.841367:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313871.908512:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313872.864796:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313872.928845:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313873.888795:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313873.952955:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313874.912770:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313874.976870:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313875.937559:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313876.007578:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761313876.704887:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313876.704894:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761313876.704897:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761313876.704907:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2119 00000100:00080000:0.0:1761313876.704910:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761313876.704917:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761313876.704925:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761313876.704939:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e00b5e9d x1846869307470528/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761313876.704957:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313876.704959:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313876.704995:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307470400:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313876.705072:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1761313876.705077:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307470336:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313876.705099:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307470464:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313876.705214:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307470528:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761313876.712539:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307470336:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313876.713433:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307470400:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313876.713462:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307470464:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761313876.713965:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313876.713971:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313876.713977:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307470528:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761313876.960798:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313877.028140:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313877.984783:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313878.048797:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313879.008931:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313879.076405:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313880.032870:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313880.096823:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313881.056766:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313881.123048:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761313881.824840:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313881.824848:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761313881.824851:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761313881.824860:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2124 00000100:00080000:0.0:1761313881.824863:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761313881.824869:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761313881.824878:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761313881.824892:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000034a6cd79 x1846869307470784/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761313881.824920:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313881.824923:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313881.824947:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307470656:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313881.825032:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1761313881.825039:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307470592:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313881.825062:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307470720:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313881.825345:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307470784:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761313881.833168:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307470592:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313881.833187:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307470720:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313881.833194:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307470656:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761313881.836623:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313881.836672:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313881.836682:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307470784:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761313882.080763:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313882.144974:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313883.104849:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313883.168831:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313884.132254:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313884.192874:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313885.152764:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313885.216863:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313886.177981:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313886.244525:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761313886.944934:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313886.944941:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761313886.944959:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761313886.944969:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2129 00000100:00080000:0.0:1761313886.944972:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761313886.944977:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761313886.944986:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761313886.945001:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000034a6cd79 x1846869307471040/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761313886.945023:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313886.945025:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313886.945047:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307470912:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313886.945077:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307471040:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761313886.945128:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1761313886.945136:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307470848:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313886.945162:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307470976:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313886.949750:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307470912:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761313886.950992:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313886.950999:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313886.951006:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307471040:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761313886.952138:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307470848:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313886.952154:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307470976:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313887.200757:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313887.265742:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313888.225932:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313888.293671:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313889.250885:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313889.314187:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313890.273253:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313890.336847:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313891.297024:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313891.360870:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761313892.064907:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313892.064916:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761313892.064920:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761313892.064932:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2135 00000100:00080000:3.0:1761313892.064935:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761313892.064942:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761313892.064971:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761313892.064988:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f597cda6 x1846869307471296/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761313892.065013:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313892.065017:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761313892.065045:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307471168:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313892.065226:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f597cda6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307471296:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761313892.065585:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307471104:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313892.069014:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f364ec74 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307471232:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313892.073564:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307471168:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761313892.073667:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761313892.073672:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761313892.073676:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f597cda6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307471296:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761313892.076576:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307471104:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313892.084211:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f364ec74 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307471232:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313892.321557:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313892.384887:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313893.344798:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313893.408877:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313894.368833:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761313894.433292:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313895.392779:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313895.456967:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313896.417407:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313896.481546:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313897.184880:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307471360:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761313897.185727:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000038534f0b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307471424:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761313897.195004:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313897.195011:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761313897.195014:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761313897.195024:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2140 00000100:00080000:3.0:1761313897.195027:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761313897.195033:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761313897.195044:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761313897.195059:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000bfffbafe x1846869307471552/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761313897.195084:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313897.195087:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761313897.195130:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761313897.195142:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307471488:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313897.195267:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307471552:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761313897.195486:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000038534f0b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307471424:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761313897.195601:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307471360:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313897.197962:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307471488:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761313897.199251:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761313897.199258:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761313897.199265:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307471552:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761313897.440765:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313897.504957:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313898.465298:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313898.531272:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313899.488813:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313899.554550:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313900.513580:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313900.576883:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313901.537302:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313901.604570:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761313902.307105:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313902.307114:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761313902.307118:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761313902.307411:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2145 00000100:00080000:2.0:1761313902.307416:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761313902.307423:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761313902.307436:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761313902.307458:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000011b5c99f x1846869307471808/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761313902.307482:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313902.307485:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761313902.307517:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b3aaddc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307471680:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313902.307704:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000011b5c99f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307471808:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761313902.307784:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e662bceb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307471616:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313902.309210:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ffb03e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307471744:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761313902.314044:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b3aaddc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307471680:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313902.316161:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e662bceb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307471616:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313902.316183:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ffb03e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307471744:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761313902.316415:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761313902.316421:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761313902.316426:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000011b5c99f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307471808:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761313902.561206:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313902.626007:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313903.584875:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313903.649025:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313904.608849:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313904.672932:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313905.633278:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313905.698291:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313906.656999:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313906.722127:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761313907.425462:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313907.425488:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761313907.425493:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761313907.425505:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2150 00000100:00080000:0.0:1761313907.425508:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761313907.425515:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761313907.425526:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761313907.425543:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e00b5e9d x1846869307472064/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761313907.425570:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313907.425573:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761313907.425677:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307472064:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761313907.426533:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307471872:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313907.426606:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307472000:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313907.426711:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761313907.426720:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307471936:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761313907.435133:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761313907.435154:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761313907.435170:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307472064:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761313907.435466:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307471936:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313907.435481:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307471872:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313907.437521:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307472000:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313907.680836:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313907.747086:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313908.704781:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313908.768859:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313909.729746:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313909.793537:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313910.752796:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313910.817719:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313911.776745:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313911.840936:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761313912.544875:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313912.544896:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761313912.544899:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761313912.544909:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2155 00000100:00080000:3.0:1761313912.544912:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761313912.544919:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761313912.544928:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761313912.544943:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000101f852a x1846869307472320/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761313912.544978:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313912.544982:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761313912.545022:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000038534f0b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307472192:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313912.545031:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000101f852a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307472320:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761313912.545138:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307472128:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761313912.549241:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000626f98db pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307472256:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313912.551030:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000038534f0b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307472192:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761313912.551127:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761313912.551132:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761313912.551136:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000101f852a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307472320:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761313912.555974:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307472128:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761313912.556005:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000626f98db pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307472256:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313912.800852:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313912.864906:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313913.824921:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313913.889089:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313914.848839:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313914.914942:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313915.874488:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313915.936859:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313916.896793:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313916.963916:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761313917.664885:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313917.664903:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761313917.664906:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761313917.664916:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2160 00000100:00080000:2.0:1761313917.664919:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761313917.664925:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761313917.664935:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761313917.664949:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000e1be9a7 x1846869307472576/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761313917.664971:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313917.664999:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761313917.665034:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ee08055d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307472448:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313917.665191:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b3aaddc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307472384:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313917.665265:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e1be9a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307472576:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761313917.666609:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008a641e2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307472512:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313917.670951:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ee08055d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307472448:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313917.671584:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b3aaddc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307472384:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313917.671619:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008a641e2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307472512:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761313917.672710:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761313917.672734:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761313917.672742:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e1be9a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307472576:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761313917.922020:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313917.984867:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313918.944800:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313919.008867:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313919.968818:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313920.032942:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313920.992818:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313921.058311:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313922.016789:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313922.082283:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761313922.786468:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313922.786487:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761313922.786492:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761313922.786518:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2165 00000100:00100000:1.0:1761313922.786519:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307472640:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761313922.786524:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761313922.786534:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761313922.786547:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761313922.786568:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000fe4020c4 x1846869307472832/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761313922.786594:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313922.786619:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761313922.786674:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307472768:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313922.786703:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307472704:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313922.787049:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307472832:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761313922.794346:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307472640:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313922.794859:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307472704:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313922.794982:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307472768:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761313922.801616:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761313922.801629:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761313922.801651:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307472832:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761313923.041296:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313923.107409:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313924.064788:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313924.130144:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313925.089449:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313925.152951:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313926.112802:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313926.178683:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313927.136794:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313927.201460:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313927.905091:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000038534f0b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307472896:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761313927.905627:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761313927.905673:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307472960:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761313927.908039:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313927.908049:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761313927.908053:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761313927.908066:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2170 00000100:00080000:3.0:1761313927.908081:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761313927.908088:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761313927.908102:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761313927.908123:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000cdcc674d x1846869307473088/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761313927.908151:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313927.908155:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761313927.908438:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f133bb2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307473024:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313927.908589:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cdcc674d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307473088:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761313927.910875:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307472960:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761313927.910901:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000038534f0b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307472896:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761313927.917887:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f133bb2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307473024:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761313927.920467:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761313927.920481:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761313927.920493:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cdcc674d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307473088:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761313928.161177:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313928.228763:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313929.186573:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313929.249452:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313930.208780:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313930.272797:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313931.232791:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313931.297363:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313932.257511:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313932.321556:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313933.025851:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307473152:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761313933.028023:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313933.028033:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761313933.028038:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761313933.028050:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2175 00000100:00080000:0.0:1761313933.028054:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761313933.028061:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761313933.028074:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761313933.028093:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007dcceb4a x1846869307473344/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761313933.028124:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313933.028128:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761313933.028263:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307473216:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313933.028373:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761313933.028379:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307473280:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313933.028494:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307473344:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761313933.030940:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307473152:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761313933.040097:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761313933.040113:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761313933.040126:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307473344:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761313933.040182:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307473216:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313933.040204:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307473280:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313933.281253:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313933.344862:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313934.305976:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313934.369968:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313935.329477:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313935.393204:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313936.352948:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313936.416888:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313937.376795:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313937.440893:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313938.145010:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307473408:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761313938.147979:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313938.148008:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761313938.148012:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761313938.148026:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2181 00000100:00080000:0.0:1761313938.148030:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761313938.148038:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000100:00100000:1.0:1761313938.148044:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307473472:192.168.203.151@tcp:400:kworker/u8:0.0 00000008:00100000:0.0:1761313938.148052:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761313938.148077:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000028de4b9d x1846869307473600/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761313938.148106:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313938.148111:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313938.148137:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307473536:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313938.148215:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307473408:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761313938.148580:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307473600:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00080000:2.0:1761313938.150353:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761313938.150375:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761313938.150385:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307473600:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761313938.150707:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307473472:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313938.150742:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307473536:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313938.400798:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313938.468193:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313939.424785:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313939.488888:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313940.448834:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313940.513725:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313941.472801:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313941.538048:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313942.496807:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313942.561895:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761313943.265085:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313943.265094:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761313943.265098:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761313943.265111:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2186 00000100:00080000:3.0:1761313943.265116:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761313943.265124:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761313943.265136:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761313943.265154:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000004e00d332 x1846869307473856/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761313943.265175:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307473664:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761313943.265184:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313943.265189:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761313943.265238:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307473728:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761313943.265390:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307473792:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313943.265452:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004e00d332 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307473856:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761313943.270681:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307473664:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313943.270682:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307473728:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761313943.270711:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307473792:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761313943.270741:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313943.270746:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313943.270751:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004e00d332 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307473856:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761313943.523184:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313943.584962:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313944.544796:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313944.609506:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313945.568769:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313945.633072:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313946.592801:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313946.656907:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313947.616798:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313947.682037:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761313948.384937:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004e00d332 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307473920:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761313948.386276:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307473984:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761313948.389365:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761313948.389394:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307474048:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761313948.396880:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313948.396894:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761313948.396899:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761313948.396912:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2191 00000100:00080000:3.0:1761313948.396916:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761313948.396923:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761313948.396939:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761313948.396980:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000039277a44 x1846869307474112/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761313948.397026:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313948.397029:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761313948.398148:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004e00d332 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307473920:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313948.398235:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307474112:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761313948.398434:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307474048:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313948.398445:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307473984:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761313948.404878:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761313948.404896:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761313948.404909:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307474112:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761313948.640828:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313948.706253:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313949.666207:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313949.730487:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313950.688815:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313950.755351:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313951.712797:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313951.778804:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313952.736986:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313952.801401:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761313953.506086:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313953.506096:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761313953.506100:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761313953.506113:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2196 00000100:00080000:0.0:1761313953.506117:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761313953.506125:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761313953.506138:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761313953.506160:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000028de4b9d x1846869307474368/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761313953.506186:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313953.506189:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313953.506222:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307474240:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313953.506349:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1761313953.506355:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307474176:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313953.506380:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307474304:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313953.507303:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307474368:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761313953.519785:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307474240:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313953.519835:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307474176:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313953.519847:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307474304:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761313953.520127:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313953.520134:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313953.520156:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307474368:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761313953.761403:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313953.825190:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313954.784777:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313954.848980:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313955.808801:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313955.877355:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313956.832817:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313956.896859:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313957.857270:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313957.920940:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761313958.625792:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1761313958.625797:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ee08055d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307474432:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761313958.625802:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761313958.625805:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761313958.625811:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2201 00000100:00080000:2.0:1761313958.625813:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761313958.625820:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761313958.625831:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761313958.625847:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006e3466f5 x1846869307474624/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761313958.625869:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313958.625872:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761313958.625899:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002292ab57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307474496:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313958.625919:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000015f165e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307474560:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313958.626023:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e3466f5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307474624:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761313958.630809:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000015f165e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307474560:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313958.630834:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ee08055d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307474432:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761313958.630952:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002292ab57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307474496:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761313958.631716:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761313958.631722:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761313958.631728:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e3466f5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307474624:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761313958.880784:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313958.945491:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313959.908812:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313959.971694:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313960.928808:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313960.992867:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313961.952811:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313962.017682:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313962.977702:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313963.041078:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761313963.744910:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307474688:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313963.749411:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307474688:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761313963.750379:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313963.750385:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761313963.750389:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761313963.750396:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2206 00000100:00080000:0.0:1761313963.750399:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761313963.750404:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761313963.750413:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761313963.750428:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000057b18867 x1846869307474880/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1761313963.750500:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307474880:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761313963.751551:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307474752:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761313963.751614:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761313963.751619:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307474816:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761313963.752024:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313963.752029:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313963.753521:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307474816:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761313963.754525:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313963.754532:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313963.754538:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307474880:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761313963.754619:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307474752:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313964.000797:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313964.065030:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313965.025179:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313965.091176:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313966.051770:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313966.119962:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313967.072875:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313967.136834:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313968.097035:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313968.160829:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761313968.865020:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1761313968.865020:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002292ab57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307474944:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761313968.865026:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761313968.865043:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761313968.865049:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2211 00000100:00080000:2.0:1761313968.865051:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761313968.865057:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761313968.865065:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761313968.865080:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ae09c97d x1846869307475136/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761313968.865102:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313968.865105:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761313968.865117:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b2298797 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307475072:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761313968.865127:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eca34e1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307475008:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313968.865202:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ae09c97d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307475136:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761313968.868429:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002292ab57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307474944:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313968.868460:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b2298797 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307475072:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761313968.870635:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761313968.870669:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761313968.870678:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ae09c97d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307475136:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761313968.871596:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eca34e1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307475008:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313969.120948:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313969.185895:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313970.144758:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313970.211735:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313971.168805:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313971.232925:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313972.193116:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313972.256908:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313973.217138:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313973.280924:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761313973.987832:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313973.987842:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761313973.987846:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761313973.987860:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2216 00000100:00080000:0.0:1761313973.987864:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761313973.987872:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761313973.987886:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761313973.987906:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007dcceb4a x1846869307475392/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761313973.987930:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313973.987933:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761313973.988028:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307475392:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761313973.988322:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307475200:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313973.990295:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307475328:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313973.990360:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761313973.990366:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307475264:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761313974.004117:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307475328:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761313974.004548:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313974.004553:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761313974.004557:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307475264:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313974.004561:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307475392:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761313974.004575:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307475200:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313974.243027:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313974.305821:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313975.264767:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313975.329542:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313976.288811:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313976.357070:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313977.312968:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313977.378680:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313978.336763:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313978.402983:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761313979.105941:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313979.105948:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761313979.105951:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761313979.105960:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2222 00000100:00080000:3.0:1761313979.105963:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761313979.105968:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761313979.105989:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761313979.106003:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000cdcc674d x1846869307475648/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761313979.106029:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313979.106033:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761313979.106091:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307475520:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313979.106178:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1761313979.106184:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307475456:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313979.106204:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307475584:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761313979.106552:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cdcc674d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307475648:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00080000:0.0:1761313979.113565:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761313979.113576:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761313979.113586:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cdcc674d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307475648:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761313979.113663:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307475456:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313979.113683:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307475584:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313979.113693:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307475520:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761313979.361198:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313979.427517:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313980.385415:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313980.448871:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313981.409120:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313981.472974:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313982.432853:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313982.497555:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313983.457716:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313983.520875:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313984.225094:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eca34e1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307475712:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761313984.226163:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313984.226168:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761313984.226171:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761313984.226177:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2227 00000100:00080000:2.0:1761313984.226180:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761313984.226185:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761313984.226193:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761313984.226205:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e9376149 x1846869307475904/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761313984.226272:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e9376149 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307475904:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00080000:2.0:1761313984.227698:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761313984.227705:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761313984.227745:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aea3302e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307475776:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313984.228301:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039978a4e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307475840:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313984.232194:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eca34e1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307475712:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:02020000:3.0:1761313984.244069:0:2262:0:(client.c:1361:ptlrpc_check_status()) 11-0: lustre-OST0000-osc-ffff951bc4d8c800: operation ost_connect to node 192.168.203.151@tcp failed: rc = -19 00000100:00100000:2.0:1761313984.244493:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aea3302e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307475776:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761313984.244554:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039978a4e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307475840:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761313984.249296:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313984.249306:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313984.249317:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e9376149 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307475904:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761313984.480772:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313984.546074:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313985.504815:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313985.569029:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313986.528799:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313986.592882:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313987.552806:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313987.617058:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313988.576809:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313988.640882:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761313989.345155:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313989.345164:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761313989.345168:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761313989.345180:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2232 00000100:00080000:3.0:1761313989.345184:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761313989.345190:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761313989.345202:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761313989.345219:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f133bb2e x1846869307476160/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761313989.345302:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f133bb2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307476160:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00080000:3.0:1761313989.345436:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313989.345440:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761313989.346951:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307475968:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313989.347126:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307476032:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761313989.347975:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307476096:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761313989.358892:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307475968:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761313989.358921:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307476096:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761313989.359388:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313989.359399:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313989.359410:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f133bb2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307476160:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761313989.359528:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307476032:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313989.600834:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313989.669214:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313990.624792:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313990.689088:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313991.648907:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313991.713309:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313992.672755:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313992.737984:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313993.696794:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313993.763049:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761313994.464886:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313994.464895:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761313994.464899:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761313994.464913:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2237 00000100:00080000:3.0:1761313994.464917:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761313994.464925:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761313994.464936:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761313994.464955:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000004505177d x1846869307476416/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761313994.464982:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761313994.464985:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313994.465184:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307476416:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761313994.465364:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307476224:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761313994.465409:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000038534f0b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307476352:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761313994.465442:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761313994.465448:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f133bb2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307476288:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761313994.469609:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307476224:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761313994.470492:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761313994.470500:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761313994.470507:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307476416:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761313994.471318:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f133bb2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307476288:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761313994.471335:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000038534f0b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307476352:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313994.720958:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313994.785217:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313995.746469:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313995.808932:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313996.769461:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313996.833064:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313997.792786:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313997.856900:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761313998.817496:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761313998.882459:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761313999.589191:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313999.589201:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761313999.589205:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761313999.589217:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2242 00000100:00080000:0.0:1761313999.589221:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761313999.589228:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761313999.589240:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761313999.589279:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000fe4020c4 x1846869307476672/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761313999.589307:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761313999.589311:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761313999.589348:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307476544:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313999.589362:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307476480:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761313999.589508:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307476672:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761313999.589513:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307476608:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313999.592970:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307476544:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761313999.593029:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307476608:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761313999.605708:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761313999.605724:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761313999.605756:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307476672:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761313999.605873:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307476480:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761313999.844835:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761313999.904830:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314000.868249:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314000.928990:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314001.888789:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314001.952992:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314002.912803:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314002.978241:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314003.936803:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314004.001702:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761314004.704923:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307476736:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761314004.706530:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314004.706539:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761314004.706543:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761314004.706553:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2247 00000100:00080000:0.0:1761314004.706557:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00100000:1.0:1761314004.706573:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307476864:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761314004.706576:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761314004.706589:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761314004.706606:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000005a420586 x1846869307476928/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761314004.706631:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314004.706635:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761314004.706650:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761314004.706656:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307476800:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761314004.707394:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307476928:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00080000:3.0:1761314004.714674:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761314004.714689:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761314004.714703:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307476928:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761314004.714833:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307476800:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761314004.714854:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307476864:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761314004.714865:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307476736:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761314004.961541:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314005.025116:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314005.986466:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314006.049822:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314007.012681:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314007.074239:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314008.032735:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314008.096918:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314009.057044:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314009.122344:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761314009.824843:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314009.824851:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761314009.824855:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761314009.824880:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2252 00000100:00080000:0.0:1761314009.824883:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761314009.824889:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761314009.824900:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761314009.824916:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b5a39135 x1846869307477184/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761314009.824937:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314009.824941:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761314009.825043:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307477184:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761314009.825054:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307476992:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761314009.825168:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307477120:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761314009.825229:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761314009.825238:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307477056:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761314009.833670:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307477120:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761314009.833713:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307476992:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761314009.833953:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761314009.833959:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761314009.833964:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307477184:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761314009.834045:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307477056:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761314010.080785:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314010.145420:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314011.107472:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314011.168871:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314012.129058:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314012.194972:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314013.152909:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314013.217521:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314014.177016:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314014.240987:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761314014.944920:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314014.944931:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761314014.944935:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761314014.944948:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2257 00000100:00080000:0.0:1761314014.944952:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761314014.944960:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761314014.944972:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761314014.944992:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b5a39135 x1846869307477440/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761314014.945019:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314014.945022:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761314014.945259:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307477248:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761314014.945691:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307477440:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761314014.948011:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307477376:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761314014.953700:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761314014.953716:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761314014.953729:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307477440:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761314014.959544:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307477248:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761314014.959576:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761314014.959589:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307477312:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761314014.970553:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307477376:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761314014.970594:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307477312:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761314015.200803:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314015.267862:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314016.224806:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314016.291140:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314017.248806:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314017.312917:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314018.272913:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314018.337104:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314019.296763:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314019.360867:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761314020.065212:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314020.065221:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761314020.065225:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761314020.065239:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2263 00000100:00080000:3.0:1761314020.065242:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761314020.065279:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761314020.065310:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761314020.065331:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000bfffbafe x1846869307477696/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761314020.065361:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314020.065365:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761314020.065468:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000626f98db pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307477504:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761314020.065597:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000101f852a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307477632:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761314020.065687:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761314020.065696:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307477568:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761314020.065702:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307477696:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761314020.071455:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307477568:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761314020.071479:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000101f852a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307477632:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761314020.072170:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000626f98db pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307477504:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761314020.072179:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761314020.072187:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761314020.072193:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307477696:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761314020.320812:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314020.384892:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314021.344785:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314021.409870:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314022.368779:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314022.433445:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314023.393629:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314023.456931:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314024.416802:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314024.481822:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314025.186289:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307477760:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761314025.187849:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f364ec74 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307477824:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761314025.188175:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761314025.188182:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307477888:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761314025.189793:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314025.189806:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761314025.189810:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761314025.189821:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2268 00000100:00080000:3.0:1761314025.189825:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761314025.189832:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761314025.189847:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761314025.189882:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f597cda6 x1846869307477952/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761314025.189900:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314025.189904:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761314025.190580:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f597cda6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307477952:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761314025.196616:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307477760:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761314025.198854:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761314025.198868:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761314025.198879:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f597cda6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307477952:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761314025.201770:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307477888:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761314025.201793:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f364ec74 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307477824:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761314025.442377:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314025.504805:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314026.471691:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314026.528932:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314027.488811:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314027.556020:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314028.512797:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314028.577688:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314029.536807:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314029.605038:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761314030.305056:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307478016:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761314030.305394:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761314030.305402:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307478080:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761314030.305755:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314030.305761:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761314030.305765:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761314030.305772:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2273 00000100:00080000:0.0:1761314030.305775:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761314030.305781:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761314030.305794:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761314030.305813:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000bcaee194 x1846869307478208/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1761314030.305889:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307478208:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00080000:0.0:1761314030.306381:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314030.306386:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761314030.306421:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761314030.306452:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307478144:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761314030.308870:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:0.0:1761314030.308875:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307478144:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761314030.308881:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761314030.308885:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307478208:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761314030.308912:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307478080:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761314030.308923:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307478016:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761314030.560803:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314030.629586:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314031.586538:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314031.648963:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314032.608882:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314032.672956:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314033.633422:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314033.699841:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314034.656942:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314034.727046:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761314035.424992:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314035.425002:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761314035.425015:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761314035.425028:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2278 00000100:00080000:1.0:1761314035.425032:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761314035.425039:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761314035.425052:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761314035.425072:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a1719d14 x1846869307478464/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761314035.425097:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314035.425101:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761314035.425188:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307478464:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761314035.427268:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307478272:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761314035.431292:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307478400:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761314035.431409:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761314035.431417:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307478336:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761314035.432448:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761314035.432455:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761314035.432461:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307478464:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761314035.435910:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307478400:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761314035.436126:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307478336:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761314035.436138:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307478272:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761314035.680887:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314035.746129:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314036.704764:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314036.773698:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314037.728767:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314037.792858:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314038.752816:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314038.817278:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314039.776782:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314039.840923:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761314040.546282:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314040.546289:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761314040.546293:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761314040.546303:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2283 00000100:00080000:0.0:1761314040.546306:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761314040.546312:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761314040.546321:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761314040.546336:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007a68d9a8 x1846869307478720/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761314040.546364:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314040.546367:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761314040.546487:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307478720:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761314040.548390:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307478592:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761314040.548480:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1761314040.548488:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307478528:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761314040.548514:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307478656:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761314040.560182:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307478656:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761314040.560202:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307478592:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761314040.561426:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307478528:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761314040.561510:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761314040.561515:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761314040.561520:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307478720:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761314040.800810:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314040.864885:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314041.824798:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314041.894438:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314042.850208:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314042.913392:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314043.873791:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314043.940424:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314044.896812:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314044.960861:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761314045.664964:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314045.664972:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761314045.664975:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761314045.664985:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2288 00000100:00080000:1.0:1761314045.664989:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761314045.664995:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761314045.665005:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761314045.665021:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a1719d14 x1846869307478976/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761314045.665049:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314045.665079:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761314045.665102:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307478848:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761314045.665111:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307478976:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761314045.665197:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1761314045.665204:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307478784:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761314045.665231:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307478912:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761314045.673170:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307478784:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761314045.673170:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761314045.673180:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761314045.673188:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307478976:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761314045.673197:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307478912:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761314045.673211:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307478848:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761314045.921763:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314045.989106:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314046.944711:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314047.008851:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314047.968733:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314048.036272:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314048.992787:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314049.056834:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314050.016778:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314050.080945:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761314050.785126:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314050.785132:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761314050.785135:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761314050.785144:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2293 00000100:00080000:0.0:1761314050.785147:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761314050.785152:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761314050.785161:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761314050.785173:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a24c5c6a x1846869307479232/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761314050.785193:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314050.785196:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761314050.785215:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307479104:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761314050.785289:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1761314050.785294:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307479040:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761314050.785311:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307479168:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761314050.785530:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307479232:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761314050.795239:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307479104:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761314050.795273:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307479040:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761314050.795285:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307479168:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761314050.796959:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761314050.796967:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761314050.796974:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307479232:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761314051.040772:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314051.109531:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314052.065228:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314052.133305:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314053.088772:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314053.152863:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314054.113819:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314054.177614:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314055.136802:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314055.201316:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761314055.904893:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00080000:0.0:1761314055.904904:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314055.904912:0:8:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761314055.904916:0:8:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1761314055.904929:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307479360:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761314055.904944:0:8:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2298 00000100:00080000:0.0:1761314055.904948:0:8:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761314055.904955:0:8:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761314055.904968:0:8:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761314055.904991:0:8:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000028de4b9d x1846869307479488/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761314055.905049:0:8:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314055.905053:0:8:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761314055.905060:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307479296:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761314055.905100:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761314055.905107:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307479424:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761314055.906948:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307479488:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00080000:3.0:1761314055.913358:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761314055.913369:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761314055.913378:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307479488:192.168.203.151@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761314055.918779:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307479424:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761314055.918799:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307479360:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761314055.918808:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307479296:192.168.203.151@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761314056.165300:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314056.224867:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314057.184774:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314057.248948:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314058.209156:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314058.272873:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314059.237291:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314059.296865:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314060.257633:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314060.320929:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314061.025512:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039978a4e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307479552:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314061.026452:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aea3302e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307479616:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761314061.026545:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1761314061.026548:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ba5c8e77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307479680:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761314061.026552:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761314061.026558:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761314061.026565:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2303 00000100:00080000:2.0:1761314061.026568:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761314061.026575:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761314061.026603:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761314061.026622:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000001426be00 x1846869307479744/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761314061.026664:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314061.026669:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761314061.027020:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001426be00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307479744:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314061.032276:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aea3302e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307479616:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314061.032448:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039978a4e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307479552:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314061.038868:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ba5c8e77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307479680:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761314061.042024:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761314061.042038:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761314061.042048:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001426be00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307479744:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761314061.280804:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314061.345047:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314062.304765:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314062.371079:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314063.331743:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314063.393139:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314064.353750:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314064.418301:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314065.376804:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314065.446366:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314066.146192:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761314066.146214:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307479808:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314066.146321:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e58cc722 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307479872:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761314066.146360:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314066.146364:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761314066.146367:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761314066.146372:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2309 00000100:00080000:3.0:1761314066.146375:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761314066.146394:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000100:00100000:3.0:1761314066.146401:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761314066.146407:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000041c8b6cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307479936:192.168.203.151@tcp:400:kworker/u8:1.0 00000008:00100000:3.0:1761314066.146437:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761314066.146453:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007673f690 x1846869307480000/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761314066.146482:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314066.146485:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761314066.146744:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007673f690 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307480000:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00080000:1.0:1761314066.152345:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761314066.152355:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761314066.152364:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007673f690 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307480000:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761314066.152421:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307479808:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314066.152450:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000041c8b6cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307479936:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314066.152458:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e58cc722 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307479872:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314066.400757:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314066.465404:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314067.427517:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314067.489067:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314068.448751:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314068.513882:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314069.473184:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314069.536891:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314070.496944:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314070.564166:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761314071.264877:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314071.264885:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761314071.264900:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761314071.264922:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2314 00000100:00080000:3.0:1761314071.264927:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761314071.264937:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000100:00100000:2.0:1761314071.264952:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e58cc722 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307480064:192.168.203.151@tcp:400:kworker/u8:1.0 00000008:00100000:3.0:1761314071.264953:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761314071.265002:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000cc6dd8ca x1846869307480256/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761314071.265027:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314071.265032:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761314071.265071:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307480192:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314071.265116:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761314071.265117:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cc6dd8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307480256:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314071.265138:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000041c8b6cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307480128:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314071.270031:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307480192:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761314071.276055:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761314071.276067:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761314071.276076:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc6dd8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307480256:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314071.276537:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000041c8b6cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307480128:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314071.276549:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e58cc722 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307480064:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314071.522023:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314071.585380:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314072.544818:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314072.609585:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314073.568821:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314073.634382:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314074.592800:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314074.656889:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314075.616890:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314075.680926:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761314076.384881:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314076.384890:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761314076.384893:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761314076.384903:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2319 00000100:00080000:0.0:1761314076.384906:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761314076.384912:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761314076.384921:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761314076.384934:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000028de4b9d x1846869307480512/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1761314076.384985:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307480512:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00080000:0.0:1761314076.385404:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314076.385422:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761314076.385699:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307480320:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314076.385733:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307480448:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314076.385766:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761314076.385770:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307480384:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314076.392584:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307480384:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314076.392607:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307480320:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314076.392619:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307480448:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761314076.393254:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761314076.393262:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761314076.393268:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307480512:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761314076.640792:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314076.706401:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314077.665591:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314077.736172:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314078.688889:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314078.753571:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314079.713446:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314079.776900:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314080.736761:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314080.804830:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761314081.506486:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314081.506494:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761314081.506498:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761314081.506508:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2324 00000100:00080000:0.0:1761314081.506511:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00100000:1.0:1761314081.506518:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307480576:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761314081.506519:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761314081.506531:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761314081.506547:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000fe4020c4 x1846869307480768/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761314081.506575:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314081.506578:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761314081.506607:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307480704:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314081.506618:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307480640:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314081.506678:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307480768:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00080000:3.0:1761314081.509190:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761314081.509198:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761314081.509207:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307480768:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761314081.509950:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307480640:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314081.510272:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307480576:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314081.510284:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307480704:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314081.760760:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314081.824948:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314082.785779:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314082.848882:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314083.808921:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314083.873689:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314084.833004:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314084.898877:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314085.856837:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314085.920904:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761314086.625272:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314086.625283:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761314086.625287:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761314086.625300:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2329 00000100:00080000:3.0:1761314086.625305:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761314086.625312:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761314086.625325:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761314086.625345:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000009911ad3 x1846869307481024/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761314086.625377:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314086.625381:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761314086.625433:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000009911ad3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307481024:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314086.625542:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307480832:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314086.625582:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d32bed84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307480960:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314086.625610:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761314086.625616:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307480896:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761314086.634408:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761314086.634426:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761314086.634439:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000009911ad3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307481024:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314086.634716:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307480896:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314086.634743:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307480832:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314086.634756:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d32bed84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307480960:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314086.881083:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314086.945668:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314087.904750:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314087.968909:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314088.929838:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314088.992895:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314089.952872:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314090.019501:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314090.976799:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314091.041097:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761314091.746704:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1761314091.746713:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aea3302e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307481088:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761314091.746729:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761314091.746733:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761314091.746738:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2334 00000100:00080000:2.0:1761314091.746741:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761314091.746758:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761314091.746771:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761314091.746789:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f220504f x1846869307481280/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761314091.746822:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314091.746827:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761314091.746849:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307481152:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314091.746855:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f220504f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307481280:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761314091.747527:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000039eebc2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307481216:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314091.756429:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307481152:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761314091.759562:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761314091.759589:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761314091.759609:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f220504f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307481280:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761314091.760968:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aea3302e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307481088:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314091.762325:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000039eebc2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307481216:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314092.001622:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314092.066000:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314093.024820:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314093.091247:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314094.048824:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314094.113246:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314095.078979:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314095.139883:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314096.097095:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314096.160893:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761314096.864942:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314096.864961:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761314096.864965:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761314096.864975:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2339 00000100:00080000:3.0:1761314096.864978:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761314096.864984:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761314096.864995:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761314096.865010:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000009911ad3 x1846869307481536/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761314096.865035:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314096.865038:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761314096.865058:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307481408:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314096.865152:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1761314096.865158:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d32bed84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307481344:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314096.865180:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307481472:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761314096.865273:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000009911ad3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307481536:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761314096.867696:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307481408:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761314096.871627:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761314096.871657:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761314096.871666:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000009911ad3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307481536:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761314096.871747:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d32bed84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307481344:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314096.871761:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307481472:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314097.120896:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314097.188947:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314098.145848:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314098.208973:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314099.168774:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314099.234826:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314100.192773:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314100.256872:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314101.219124:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314101.283105:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761314101.992215:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314101.992222:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761314101.992225:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761314101.992248:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2344 00000100:00080000:2.0:1761314101.992251:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761314101.992256:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761314101.992266:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761314101.992283:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b4eac89a x1846869307481792/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761314101.992308:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314101.992311:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761314101.992339:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004592c0a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307481664:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761314101.992354:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b4eac89a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307481792:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761314101.992460:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307481600:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314101.992894:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000007136630 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307481728:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761314101.996100:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761314101.996121:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761314101.996130:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b4eac89a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307481792:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314101.996497:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004592c0a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307481664:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314101.996550:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307481600:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314101.997191:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000007136630 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307481728:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314102.241318:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314102.304851:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314103.265570:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314103.329153:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314104.288779:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314104.352882:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314105.312785:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314105.376843:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314106.336771:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314106.401315:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761314107.104874:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314107.104882:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761314107.104886:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761314107.104896:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2350 00000100:00080000:2.0:1761314107.104900:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761314107.104905:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761314107.104915:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761314107.104930:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b4eac89a x1846869307482048/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761314107.104972:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314107.104975:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761314107.105132:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004592c0a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307481856:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314107.105223:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307481984:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314107.105254:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761314107.105260:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000007136630 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307481920:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761314107.105904:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b4eac89a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307482048:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761314107.112941:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000007136630 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307481920:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314107.112960:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004592c0a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307481856:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314107.112970:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307481984:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761314107.113381:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761314107.113393:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761314107.113398:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b4eac89a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307482048:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314107.361858:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314107.425154:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314108.384767:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314108.449160:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314109.408789:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314109.472858:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314110.433085:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314110.496871:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314111.456789:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314111.523854:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761314112.224959:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314112.224969:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761314112.224973:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761314112.224985:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2355 00000100:00080000:3.0:1761314112.224989:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761314112.224997:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761314112.225009:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761314112.225028:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e58cc722 x1846869307482304/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761314112.225086:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314112.225090:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761314112.225117:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d32bed84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307482176:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314112.225234:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1761314112.225240:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307482112:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314112.225269:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307482240:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761314112.226684:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e58cc722 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307482304:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761314112.232817:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307482112:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761314112.236997:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761314112.237005:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761314112.237014:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e58cc722 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307482304:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761314112.237465:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307482240:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314112.237478:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d32bed84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307482176:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314112.480750:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314112.549432:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314113.504763:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314113.581365:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314114.528770:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314114.592853:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314115.552765:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314115.616851:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314116.576814:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314116.641922:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761314117.344914:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314117.346720:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761314117.346725:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761314117.346738:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2360 00000100:00080000:3.0:1761314117.346742:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761314117.346749:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761314117.346765:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761314117.346807:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000041c8b6cb x1846869307482560/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761314117.346855:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314117.346858:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761314117.346907:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307482432:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314117.347188:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d32bed84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307482368:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761314117.348676:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000041c8b6cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307482560:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314117.350295:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307482496:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314117.356209:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d32bed84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307482368:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314117.356239:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307482496:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314117.356337:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307482432:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761314117.356805:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761314117.356811:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761314117.356816:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000041c8b6cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307482560:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314117.600809:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314117.669005:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314118.629083:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314118.692024:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314119.652766:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314119.713057:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314120.672766:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314120.736876:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314121.696874:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314121.761681:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761314122.474020:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314122.474027:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761314122.474031:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761314122.474040:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2365 00000100:00080000:0.0:1761314122.474043:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761314122.474049:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761314122.474059:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761314122.474075:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000005b900d12 x1846869307482816/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761314122.474101:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314122.474105:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761314122.475253:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307482816:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761314122.475694:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307482624:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314122.475746:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307482752:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314122.475772:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761314122.475779:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307482688:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314122.479567:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307482688:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314122.479584:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307482624:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314122.479593:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307482752:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761314122.485704:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761314122.485715:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761314122.485741:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307482816:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761314122.721014:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314122.785325:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314123.744773:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314123.809197:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314124.769080:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314124.833918:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314125.792754:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314125.857872:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314126.816808:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314126.880829:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761314127.584898:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314127.584905:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761314127.584908:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761314127.584918:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2370 00000100:00080000:2.0:1761314127.584921:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761314127.584927:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761314127.584937:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761314127.584953:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000001426be00 x1846869307483072/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761314127.585009:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001426be00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307483072:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761314127.585170:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000039eebc2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307482880:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761314127.587418:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314127.587425:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761314127.587451:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aea3302e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307482944:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761314127.590548:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761314127.590557:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761314127.590566:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001426be00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307483072:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761314127.590735:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f220504f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307483008:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314127.591315:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000039eebc2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307482880:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314127.594835:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aea3302e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307482944:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314127.595603:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f220504f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307483008:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314127.840850:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314127.905855:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314128.864770:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314128.929071:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314129.889274:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314129.954515:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314130.912774:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314130.976897:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314131.938610:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314132.004657:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761314132.705899:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314132.705907:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761314132.705909:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761314132.705919:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2375 00000100:00080000:1.0:1761314132.705922:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761314132.705927:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761314132.705936:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761314132.705950:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000009cec3faa x1846869307483328/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761314132.706017:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307483328:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00080000:1.0:1761314132.706842:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314132.706848:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761314132.706870:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307483200:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314132.706916:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1761314132.706923:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307483136:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314132.706943:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307483264:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314132.719871:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307483136:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314132.719911:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307483264:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314132.719922:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307483200:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761314132.720031:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761314132.720036:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761314132.720040:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307483328:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761314132.960814:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314133.028460:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314133.984774:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314134.048890:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314135.008922:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314135.074482:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314136.032773:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314136.102228:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314137.059613:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314137.126830:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761314137.824989:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307483392:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314137.827085:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761314137.827136:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307483456:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761314137.827516:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314137.827523:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761314137.827528:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761314137.827534:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2380 00000100:00080000:0.0:1761314137.827537:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761314137.827543:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761314137.827556:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761314137.827574:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e00b5e9d x1846869307483584/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761314137.827590:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314137.827593:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761314137.827623:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761314137.827634:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307483520:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314137.828523:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307483584:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761314137.830457:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307483520:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314137.830561:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307483392:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314137.832767:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307483456:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:02020000:2.0:1761314137.834149:0:2262:0:(client.c:1361:ptlrpc_check_status()) 11-0: lustre-OST0000-osc-ffff951bc4d8c800: operation ost_connect to node 192.168.203.151@tcp failed: rc = -19 00000100:00080000:2.0:1761314137.839305:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761314137.839314:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761314137.839324:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307483584:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761314138.080763:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314138.144881:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314139.104764:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314139.168887:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314140.128769:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314140.193710:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314141.155328:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314141.216877:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314142.176771:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314142.240958:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314142.944995:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aea3302e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307483648:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314142.945133:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761314142.945140:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001426be00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307483712:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761314142.945619:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314142.945624:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761314142.945627:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761314142.945681:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2385 00000100:00080000:2.0:1761314142.945684:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761314142.945690:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761314142.945698:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761314142.945713:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000039978a4e x1846869307483840/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761314142.945735:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314142.945738:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761314142.945940:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039978a4e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307483840:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761314142.946922:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761314142.946942:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ba5c8e77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307483776:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314142.951500:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aea3302e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307483648:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314142.952707:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001426be00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307483712:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761314142.952861:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761314142.952867:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761314142.952872:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039978a4e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307483840:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314142.952973:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ba5c8e77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307483776:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314143.200860:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314143.265130:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314144.224816:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314144.289553:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314145.248841:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314145.313505:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314146.274451:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314146.337935:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314147.298186:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314147.361569:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761314148.064987:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314148.064997:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761314148.065002:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761314148.065017:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2391 00000100:00080000:1.0:1761314148.065021:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761314148.065030:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761314148.065046:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761314148.065066:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000009cec3faa x1846869307484096/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761314148.065097:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314148.065118:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761314148.065176:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307483968:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314148.065285:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1761314148.065291:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307483904:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314148.065314:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307484032:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314148.065432:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307484096:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761314148.076572:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307483904:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314148.076606:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307484032:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314148.076620:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307483968:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761314148.082808:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761314148.082827:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761314148.082843:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307484096:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761314148.321445:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314148.385593:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314149.345415:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314149.409048:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314150.369572:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314150.432852:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314151.394548:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314151.457280:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314152.416799:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314152.483943:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761314153.188137:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314153.188144:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761314153.188147:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761314153.188158:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2396 00000100:00080000:3.0:1761314153.188161:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761314153.188166:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761314153.188175:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761314153.188202:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007673f690 x1846869307484352/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761314153.188228:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314153.188231:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761314153.188257:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cc6dd8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307484224:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314153.188356:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1761314153.188362:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307484160:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314153.188384:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307484288:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761314153.188471:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007673f690 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307484352:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00080000:1.0:1761314153.193344:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761314153.193355:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761314153.193365:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007673f690 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307484352:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761314153.193595:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307484288:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314153.193618:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc6dd8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307484224:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314153.193922:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307484160:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314153.440992:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314153.506430:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314154.464790:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314154.528896:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314155.488851:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314155.552853:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314156.512778:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314156.577349:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314157.536788:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314157.601062:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761314158.304906:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314158.304913:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761314158.304915:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761314158.304926:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2401 00000100:00080000:2.0:1761314158.304929:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761314158.304934:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761314158.304943:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761314158.304957:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ae09c97d x1846869307484608/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761314158.304979:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314158.304982:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761314158.305022:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e9376149 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307484480:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314158.305114:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1761314158.305120:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ba5c8e77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307484416:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314158.305144:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eca34e1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307484544:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761314158.306785:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ae09c97d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307484608:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314158.311817:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eca34e1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307484544:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314158.311836:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e9376149 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307484480:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761314158.312106:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761314158.312124:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761314158.312129:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ae09c97d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307484608:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314158.317108:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ba5c8e77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307484416:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314158.561041:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314158.625259:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314159.584778:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314159.649187:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314160.608765:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314160.681944:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314161.632777:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314161.698723:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314162.656822:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314162.722863:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761314163.425503:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314163.425515:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761314163.425519:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761314163.425533:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2406 00000100:00080000:1.0:1761314163.425537:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761314163.425545:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761314163.425557:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761314163.425576:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000eb8b1f19 x1846869307484864/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761314163.425603:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314163.425607:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761314163.425688:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307484864:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761314163.425918:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307484672:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314163.428113:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307484800:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314163.428197:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761314163.428204:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307484736:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314163.432748:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307484736:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314163.432774:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307484672:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314163.432786:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307484800:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761314163.434819:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761314163.434832:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761314163.434843:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307484864:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761314163.680795:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314163.745985:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314164.707349:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314164.770552:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314165.728848:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314165.796142:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314166.752782:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314166.817444:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314167.776977:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314167.841765:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761314168.544972:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314168.544980:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761314168.544983:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761314168.544993:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2411 00000100:00080000:0.0:1761314168.545006:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761314168.545012:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761314168.545022:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761314168.545037:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e00b5e9d x1846869307485120/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761314168.545117:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307484928:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761314168.545249:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307485056:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761314168.545283:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761314168.545290:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307484992:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761314168.546215:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314168.546220:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761314168.546878:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307485120:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761314168.553481:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307484928:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761314168.553819:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761314168.553824:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761314168.553829:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307485120:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761314168.553900:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307484992:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761314168.553912:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307485056:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314168.800775:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314168.868303:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314169.824883:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314169.889935:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314170.848805:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314170.912849:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314171.874324:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314171.937480:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314172.897458:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314172.964652:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761314173.665109:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314173.665116:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761314173.665119:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761314173.665130:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2416 00000100:00080000:0.0:1761314173.665133:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761314173.665139:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761314173.665149:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761314173.665216:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000bcaee194 x1846869307485376/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761314173.665251:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314173.665255:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761314173.665283:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307485248:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314173.665324:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307485376:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761314173.665393:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1761314173.665406:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307485184:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761314173.665461:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307485312:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761314173.670386:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307485184:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761314173.670417:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307485312:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761314173.670430:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307485248:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761314173.672713:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761314173.672726:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761314173.672745:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307485376:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761314173.920796:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314173.985057:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314174.944826:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314175.008876:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314175.969541:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314176.035026:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314176.992720:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314177.057257:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314178.016883:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314178.084711:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761314178.784978:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307485440:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761314178.787052:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314178.787059:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761314178.787062:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1761314178.787063:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307485504:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761314178.787068:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2421 00000100:00080000:0.0:1761314178.787070:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761314178.787076:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761314178.787085:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761314178.787098:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ace4d025 x1846869307485632/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761314178.787140:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761314178.787146:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307485568:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314178.787175:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307485632:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00080000:0.0:1761314178.787552:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314178.787569:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761314178.790228:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307485568:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761314178.790237:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307485440:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761314178.790269:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307485504:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761314178.790428:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761314178.790434:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761314178.790439:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307485632:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761314179.040790:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314179.105441:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314180.069245:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314180.133044:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314181.088748:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314181.152988:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314182.112794:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314182.178840:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314183.136772:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314183.201262:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761314183.904844:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00080000:1.0:1761314183.904851:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314183.904858:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761314183.904862:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1761314183.904874:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307485760:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761314183.904874:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2426 00000100:00080000:1.0:1761314183.904891:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761314183.904898:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761314183.904908:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761314183.904928:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000540c3ce6 x1846869307485888/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761314183.904951:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314183.904954:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761314183.904964:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307485696:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314183.905001:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761314183.905005:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307485888:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761314183.905015:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307485824:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314183.911161:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307485696:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761314183.911397:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761314183.911407:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761314183.911415:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307485888:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761314183.913373:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307485824:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314183.913395:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307485760:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314184.160759:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314184.230916:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314185.184927:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314185.248825:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314186.210816:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314186.282127:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314187.232807:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314187.296917:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314188.258813:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314188.322464:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761314189.024912:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314189.024922:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761314189.024927:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761314189.024941:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2431 00000100:00080000:1.0:1761314189.024946:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761314189.024954:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761314189.024967:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761314189.024987:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000003871442b x1846869307486144/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761314189.025012:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314189.025016:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761314189.025097:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307486016:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314189.025229:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1761314189.025235:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307485952:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314189.025258:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307486080:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314189.026954:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307486144:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761314189.032160:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307485952:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314189.032190:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307486080:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314189.032202:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307486016:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761314189.036620:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761314189.036674:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761314189.036686:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307486144:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761314189.280799:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314189.345545:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314190.304762:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314190.368911:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314191.330469:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314191.392844:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314192.352876:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314192.416856:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314193.377842:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314193.442921:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761314194.145138:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314194.145148:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761314194.145153:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761314194.145167:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2437 00000100:00080000:1.0:1761314194.145171:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761314194.145178:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761314194.145190:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761314194.145209:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000009cec3faa x1846869307486400/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761314194.145235:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314194.145239:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761314194.145272:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307486272:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314194.145383:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1761314194.145390:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307486208:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314194.145413:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307486336:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314194.145746:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307486400:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761314194.153386:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307486208:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314194.158929:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307486336:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314194.160477:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307486272:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761314194.160697:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761314194.160704:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761314194.160709:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307486400:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761314194.404503:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314194.465921:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314195.424838:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314195.492464:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314196.449546:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314196.512883:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314197.472814:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314197.536955:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314198.496968:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314198.567576:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761314199.264901:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314199.264928:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761314199.264932:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761314199.264943:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2442 00000100:00080000:0.0:1761314199.264947:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761314199.264953:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761314199.264963:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761314199.264979:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008e59d025 x1846869307486656/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761314199.265035:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314199.265038:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761314199.265069:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307486528:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761314199.265167:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1761314199.265173:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307486464:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761314199.265191:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307486592:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314199.265859:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307486656:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761314199.274747:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307486464:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761314199.274780:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307486592:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761314199.274794:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307486528:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761314199.274823:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761314199.274852:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761314199.274856:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307486656:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314199.522393:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314199.584894:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314200.547818:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314200.610345:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314201.568742:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314201.636120:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314202.593574:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314202.657000:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314203.616797:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314203.681396:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761314204.385040:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314204.385046:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761314204.385049:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761314204.385063:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2447 00000100:00080000:1.0:1761314204.385066:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761314204.385072:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761314204.385082:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761314204.385095:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000540c3ce6 x1846869307486912/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761314204.385117:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314204.385120:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761314204.385143:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307486784:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314204.385270:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307486912:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761314204.385459:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307486720:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314204.386938:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307486848:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314204.389591:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307486784:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761314204.396213:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761314204.396223:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761314204.396232:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307486912:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761314204.396326:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307486720:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314204.396340:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307486848:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314204.640770:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314204.705668:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314205.664700:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314205.728855:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314206.688835:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314206.752880:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314207.716855:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314207.776866:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314208.736862:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314208.800873:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761314209.508080:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314209.508091:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761314209.508096:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761314209.508109:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2452 00000100:00080000:0.0:1761314209.508113:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761314209.508121:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761314209.508135:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761314209.508156:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007a68d9a8 x1846869307487168/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761314209.508180:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314209.508183:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761314209.508233:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307487040:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314209.508278:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307487168:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761314209.508358:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1761314209.508369:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307486976:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314209.508406:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307487104:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761314209.521527:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761314209.521542:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761314209.521554:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307487168:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761314209.522956:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307486976:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314209.522983:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307487104:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314209.522997:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307487040:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314209.761864:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314209.828995:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314210.785521:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314210.851338:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314211.815311:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314211.873436:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314212.832765:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314212.899273:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314213.856855:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314213.922120:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761314214.624940:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314214.624950:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761314214.624975:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761314214.624991:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2457 00000100:00080000:1.0:1761314214.624996:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761314214.625004:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761314214.625019:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761314214.625044:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a15ef099 x1846869307487424/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761314214.625075:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314214.625078:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761314214.625112:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307487296:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314214.625233:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1761314214.625238:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307487232:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314214.625261:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307487360:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314214.625500:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307487424:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761314214.632781:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307487232:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314214.632809:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307487360:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314214.632820:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307487296:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761314214.633315:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761314214.633321:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761314214.633327:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307487424:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761314214.881176:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314214.945006:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314215.905281:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314215.968924:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314216.928767:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314216.993950:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314217.952825:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314218.017629:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314218.977440:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314219.041254:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761314219.745891:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314219.745900:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761314219.745904:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761314219.745920:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2462 00000100:00080000:0.0:1761314219.745924:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761314219.745931:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761314219.745943:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761314219.745963:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000005a420586 x1846869307487680/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761314219.745996:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314219.746000:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761314219.746037:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307487488:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314219.746057:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307487680:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761314219.746152:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307487616:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761314219.746621:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761314219.746631:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307487552:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761314219.755575:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307487616:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761314219.758527:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761314219.758541:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761314219.758554:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307487680:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761314219.759450:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307487552:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761314219.759468:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307487488:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314220.000805:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314220.067624:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314221.027172:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314221.090242:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314222.052606:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314222.113184:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314223.073213:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314223.138093:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314224.096804:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314224.160961:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761314224.864959:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314224.864970:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761314224.864973:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761314224.864987:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2467 00000100:00080000:3.0:1761314224.864991:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761314224.864998:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761314224.865012:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761314224.865034:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f364ec74 x1846869307487936/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761314224.865098:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f364ec74 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307487936:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314224.866014:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307487744:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314224.866085:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307487872:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314224.866119:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761314224.866128:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cc6dd8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307487808:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761314224.866634:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314224.869333:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00080000:0.0:1761314224.873725:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761314224.873739:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761314224.873740:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc6dd8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307487808:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761314224.873744:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f364ec74 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307487936:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314224.873755:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307487872:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314224.874262:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307487744:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314225.120875:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314225.184907:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314226.145232:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314226.211173:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314227.169346:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314227.232941:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314228.192798:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314228.260494:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314229.216803:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314229.284053:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761314229.986516:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314229.986524:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761314229.986528:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761314229.986540:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2472 00000100:00080000:0.0:1761314229.986544:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761314229.986551:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761314229.986563:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761314229.986579:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000028de4b9d x1846869307488192/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761314229.986691:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307488192:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761314229.987288:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307488064:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761314229.987347:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1761314229.987372:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307488000:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761314229.987399:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307488128:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761314229.988456:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314229.988465:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761314229.992823:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307488128:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761314229.992862:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307488064:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761314229.996052:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761314229.996067:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761314229.996079:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307488192:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761314229.997830:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307488000:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314230.240833:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314230.305611:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314231.265697:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314231.329178:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314232.288835:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314232.354996:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314233.313662:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314233.377431:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314234.336864:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314234.400913:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761314235.104921:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314235.104932:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761314235.104941:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761314235.104957:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2478 00000100:00080000:0.0:1761314235.104962:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761314235.104992:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761314235.105006:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761314235.105026:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007dcceb4a x1846869307488448/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761314235.105054:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314235.105057:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761314235.105096:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307488320:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314235.105110:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307488448:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761314235.105712:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307488256:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761314235.105764:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307488384:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761314235.108072:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761314235.108094:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761314235.108125:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307488448:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761314235.108130:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307488256:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761314235.108152:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307488384:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761314235.108193:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307488320:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314235.361232:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314235.425815:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314236.384802:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314236.449299:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314237.411305:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314237.473338:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314238.432814:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314238.496875:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314239.456919:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314239.521535:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761314240.226743:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314240.226754:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761314240.226758:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761314240.226784:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2483 00000100:00080000:1.0:1761314240.226788:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761314240.226796:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761314240.226810:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761314240.226831:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000003871442b x1846869307488704/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761314240.226857:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314240.226860:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761314240.226897:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307488576:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314240.226900:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307488704:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761314240.226982:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307488512:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314240.227532:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307488640:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314240.238772:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307488512:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761314240.239196:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761314240.239202:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761314240.239207:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307488704:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761314240.239983:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307488576:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314240.240024:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307488640:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314240.481426:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314240.546447:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314241.504840:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314241.568996:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314242.529341:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314242.592842:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314243.552823:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314243.616963:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314244.576806:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314244.640905:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761314245.345142:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314245.345153:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761314245.345157:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761314245.345170:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2488 00000100:00080000:0.0:1761314245.345192:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761314245.345199:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761314245.345210:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761314245.345242:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000057b18867 x1846869307488960/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761314245.345269:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314245.345273:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761314245.345308:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307488832:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314245.345320:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307488960:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761314245.345475:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307488768:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314245.353133:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307488896:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314245.356035:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307488832:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761314245.356135:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761314245.356140:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761314245.356145:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307488960:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761314245.359092:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307488768:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314245.365698:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307488896:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314245.600837:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314245.664879:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314246.624804:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314246.690362:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314247.649442:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314247.712936:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314248.672807:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314248.739580:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314249.696796:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314249.761041:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761314250.465447:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314250.465458:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761314250.465462:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761314250.465475:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2493 00000100:00080000:1.0:1761314250.465479:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761314250.465487:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761314250.465500:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761314250.465520:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000163e3801 x1846869307489216/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761314250.465547:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314250.465551:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761314250.465586:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307489088:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761314250.465756:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1761314250.465762:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307489024:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761314250.465815:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307489152:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314250.465927:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307489216:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761314250.471231:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307489088:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761314250.472850:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307489152:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761314250.474007:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307489024:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761314250.475408:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761314250.475419:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761314250.475427:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307489216:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761314250.720803:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314250.787365:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314251.744801:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314251.810167:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314252.768962:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314252.832968:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314253.792942:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314253.857672:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314254.817009:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314254.881103:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761314255.584878:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314255.584885:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761314255.584889:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761314255.584899:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2498 00000100:00080000:3.0:1761314255.584903:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761314255.584909:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761314255.584919:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761314255.584945:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000bfffbafe x1846869307489472/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761314255.584965:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307489280:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314255.585001:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307489472:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761314255.585634:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307489408:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314255.585713:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761314255.585718:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f597cda6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307489344:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761314255.586048:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314255.586053:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761314255.597001:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f597cda6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307489344:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314255.597024:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307489280:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314255.597073:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307489408:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761314255.597080:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761314255.597092:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761314255.597101:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307489472:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761314255.840760:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314255.904879:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314256.864785:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314256.928857:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314257.888774:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314257.953006:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314258.917277:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314258.977868:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314259.936813:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314260.001871:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761314260.706159:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314260.706170:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761314260.706174:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761314260.706188:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2503 00000100:00080000:0.0:1761314260.706192:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761314260.706200:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761314260.706214:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761314260.706234:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000084f3d917 x1846869307489728/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761314260.706268:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314260.706272:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761314260.706307:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307489600:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314260.706452:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307489536:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314260.706596:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307489664:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314260.707846:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307489728:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761314260.711202:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307489536:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314260.711227:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307489664:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761314260.714805:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761314260.714820:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761314260.714832:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307489728:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761314260.716752:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307489600:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314260.961398:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314261.027138:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314261.985578:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314262.048905:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314263.009469:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314263.073165:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314264.035791:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314264.100872:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314265.061694:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314265.121605:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761314265.824897:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314265.824908:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761314265.824912:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761314265.824925:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2508 00000100:00080000:3.0:1761314265.824930:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761314265.824937:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761314265.824951:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761314265.824971:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000038534f0b x1846869307489984/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761314265.825000:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314265.825004:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761314265.825060:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000038534f0b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307489984:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314265.825105:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000626f98db pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307489792:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314265.825188:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307489920:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314265.825222:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761314265.825229:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000101f852a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307489856:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314265.828508:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000101f852a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307489856:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314265.828545:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000626f98db pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307489792:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314265.828556:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307489920:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761314265.828949:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761314265.828956:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761314265.828961:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000038534f0b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307489984:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761314266.081028:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314266.146717:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314267.104809:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314267.171194:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314268.128794:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314268.197430:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314269.152800:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314269.219009:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314270.177954:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314270.242989:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761314270.945014:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314270.945025:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761314270.945029:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761314270.945043:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2513 00000100:00080000:3.0:1761314270.945047:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761314270.945055:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761314270.945068:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761314270.945086:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ddd3c50a x1846869307490240/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761314270.945116:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314270.945120:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761314270.945216:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000038534f0b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307490048:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314270.945325:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307490176:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314270.945360:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761314270.945365:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f133bb2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307490112:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761314270.945467:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307490240:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00080000:3.0:1761314270.951053:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761314270.951067:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761314270.951079:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307490240:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314270.951177:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f133bb2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307490112:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314270.951188:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000038534f0b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307490048:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314270.951215:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307490176:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314271.200798:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314271.264920:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314272.225085:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314272.291236:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314273.248801:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314273.316594:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314274.272795:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314274.337397:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314275.303575:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314275.360884:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761314276.066817:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314276.066826:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761314276.066830:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761314276.066843:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2519 00000100:00080000:0.0:1761314276.066847:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761314276.066855:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761314276.066868:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761314276.066886:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007dcceb4a x1846869307490496/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761314276.066910:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314276.066914:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761314276.066943:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307490368:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314276.067050:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1761314276.067057:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307490304:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314276.067080:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307490432:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314276.068708:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307490496:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761314276.072835:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307490304:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314276.072884:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307490432:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314276.072896:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307490368:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761314276.073156:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761314276.073162:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761314276.073166:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307490496:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761314276.323406:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314276.384913:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314277.344817:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314277.410488:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314278.368820:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314278.433975:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314279.392844:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314279.456998:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314280.417825:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314280.480920:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761314281.185099:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761314281.185141:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307490560:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314281.185171:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761314281.185184:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307490624:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761314281.185493:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314281.185499:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761314281.185504:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761314281.185510:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2524 00000100:00080000:1.0:1761314281.185514:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761314281.185522:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761314281.185535:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761314281.185555:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000009cec3faa x1846869307490752/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761314281.185571:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314281.185575:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761314281.185589:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307490752:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761314281.185598:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307490688:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314281.188409:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307490688:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314281.188837:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307490624:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314281.191359:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307490560:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761314281.193344:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761314281.193358:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761314281.193369:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307490752:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761314281.440801:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314281.505310:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314282.464786:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314282.528920:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314283.488798:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314283.552966:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314284.512946:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314284.576901:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314285.537082:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314285.604844:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761314286.305011:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314286.305021:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761314286.305024:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761314286.305038:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2529 00000100:00080000:3.0:1761314286.305042:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761314286.305050:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761314286.305062:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761314286.305079:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000cdcc674d x1846869307491008/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761314286.305105:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314286.305108:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761314286.305193:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307490816:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314286.305307:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307490944:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314286.305348:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761314286.305354:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307490880:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761314286.305460:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cdcc674d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307491008:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314286.307904:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307490880:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761314286.311890:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761314286.311905:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761314286.311918:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cdcc674d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307491008:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314286.311998:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307490816:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314286.312023:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307490944:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314286.561107:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314286.624889:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314287.584858:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314287.649165:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314288.609985:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314288.673629:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314289.632802:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314289.696925:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314290.657332:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314290.721594:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761314291.424936:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314291.424944:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761314291.424948:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761314291.424959:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2534 00000100:00080000:1.0:1761314291.424962:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761314291.424968:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761314291.424977:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761314291.424992:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000163e3801 x1846869307491264/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761314291.425016:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314291.425020:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761314291.425045:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307491136:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314291.425153:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1761314291.425163:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307491072:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314291.425190:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307491200:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314291.425692:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307491264:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:02020000:3.0:1761314291.437230:0:2262:0:(client.c:1361:ptlrpc_check_status()) 11-0: lustre-OST0000-osc-ffff951bc4d8c800: operation ost_connect to node 192.168.203.151@tcp failed: rc = -19 00000100:00080000:3.0:1761314291.437247:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761314291.437254:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761314291.437260:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307491264:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761314291.437377:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307491072:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314291.437392:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307491200:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314291.437402:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307491136:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314291.680808:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314291.749357:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314292.704874:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314292.768873:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314293.728804:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314293.792921:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314294.757095:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314294.819860:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314295.777467:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314295.840919:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761314296.544949:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307491328:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314296.545276:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761314296.545283:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307491392:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314296.545852:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307491456:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761314296.546631:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314296.546659:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761314296.546663:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761314296.546672:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2539 00000100:00080000:1.0:1761314296.546675:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761314296.546682:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761314296.546696:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761314296.546721:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ed2f9bca x1846869307491520/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761314296.546737:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314296.546741:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761314296.546767:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307491520:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00080000:3.0:1761314296.555232:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:1.0:1761314296.555241:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307491456:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761314296.555245:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761314296.555253:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307491520:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761314296.555262:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307491392:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314296.555311:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307491328:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314296.800871:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314296.864848:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314297.824794:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314297.889589:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314298.848792:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314298.913998:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314299.872782:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314299.939559:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314300.896802:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314300.961071:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761314301.665731:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314301.665740:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761314301.665761:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761314301.665773:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2544 00000100:00080000:2.0:1761314301.665777:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761314301.665784:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761314301.665796:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761314301.665812:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b2298797 x1846869307491776/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761314301.665832:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314301.665835:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761314301.665866:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e9376149 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307491648:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314301.665976:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1761314301.665982:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ba5c8e77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307491584:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314301.666005:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eca34e1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307491712:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761314301.666795:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b2298797 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307491776:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314301.670828:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ba5c8e77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307491584:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761314301.672136:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761314301.672153:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761314301.672161:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b2298797 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307491776:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314301.672326:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eca34e1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307491712:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314301.672339:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e9376149 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307491648:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314301.921529:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314301.987054:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314302.945284:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314303.008928:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314303.968800:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314304.034365:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314304.992861:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314305.059239:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314306.016802:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314306.080941:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761314306.784841:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314306.784849:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761314306.784853:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761314306.784865:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2549 00000100:00080000:1.0:1761314306.784869:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761314306.784876:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761314306.784888:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761314306.784905:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a1719d14 x1846869307492032/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761314306.784934:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314306.784938:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761314306.784970:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307491904:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314306.785075:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1761314306.785082:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307491840:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314306.785132:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307491968:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314306.785666:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307492032:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761314306.788785:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307491840:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314306.788817:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307491968:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314306.788829:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307491904:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761314306.808375:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761314306.808397:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761314306.808409:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307492032:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761314307.041313:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314307.105686:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314308.064836:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314308.129460:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314309.089333:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314309.153537:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314310.114494:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314310.177985:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314311.137821:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314311.201418:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761314311.907771:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314311.907779:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761314311.907783:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761314311.907795:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2554 00000100:00080000:3.0:1761314311.907799:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761314311.907807:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761314311.907818:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761314311.907852:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ddd3c50a x1846869307492288/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761314311.907876:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314311.907879:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761314311.907908:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004e00d332 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307492160:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314311.908015:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1761314311.908020:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307492096:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314311.908038:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307492224:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761314311.908460:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307492288:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761314311.912968:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307492096:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314311.914548:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004e00d332 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307492160:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314311.922532:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307492224:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761314311.930989:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761314311.931005:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761314311.931018:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307492288:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761314312.160798:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314312.226104:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314313.184841:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314313.250169:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314314.208797:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314314.273238:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314315.232733:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314315.296882:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314316.256884:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314316.322527:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761314317.025020:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314317.025029:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761314317.025055:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761314317.025068:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2559 00000100:00080000:2.0:1761314317.025072:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761314317.025079:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761314317.025092:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761314317.025113:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000002292ab57 x1846869307492544/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761314317.025139:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314317.025142:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761314317.025179:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eca34e1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307492416:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314317.025348:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e9376149 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307492352:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761314317.026564:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002292ab57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307492544:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761314317.026938:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ba5c8e77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307492480:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314317.031904:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e9376149 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307492352:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314317.031938:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ba5c8e77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307492480:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761314317.032322:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761314317.032328:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761314317.032334:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002292ab57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307492544:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314317.032567:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eca34e1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307492416:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314317.281001:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314317.345554:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314318.305789:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314318.368849:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314319.328786:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314319.395352:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314320.358354:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314320.416827:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314321.378002:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314321.440859:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761314322.144927:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314322.144937:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761314322.144941:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761314322.144954:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2565 00000100:00080000:3.0:1761314322.144958:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761314322.144965:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761314322.144976:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761314322.144994:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000004505177d x1846869307492800/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761314322.145094:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307492800:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761314322.145297:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307492608:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314322.145332:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307492736:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314322.145361:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761314322.145367:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cdcc674d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307492672:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761314322.145610:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314322.145614:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00080000:1.0:1761314322.155208:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761314322.155221:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761314322.155231:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307492800:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761314322.155343:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cdcc674d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307492672:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314322.155354:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307492608:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314322.155364:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307492736:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314322.400808:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314322.464863:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314323.426410:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314323.490802:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314324.449110:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314324.513823:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314325.472808:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314325.536873:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314326.498579:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314326.562905:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761314327.265092:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314327.265098:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761314327.265112:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761314327.265120:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2570 00000100:00080000:2.0:1761314327.265124:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761314327.265129:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761314327.265138:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761314327.265152:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000015f165e6 x1846869307493056/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761314327.265215:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000015f165e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307493056:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761314327.265657:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eca34e1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307492864:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314327.265693:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ee08055d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307492992:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314327.265717:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761314327.265723:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e3466f5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307492928:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761314327.265919:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314327.265923:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00080000:1.0:1761314327.275434:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761314327.275447:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761314327.275459:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000015f165e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307493056:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761314327.275465:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e3466f5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307492928:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314327.275489:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eca34e1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307492864:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314327.275522:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ee08055d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307492992:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314327.520826:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314327.585602:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314328.547994:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314328.608976:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314329.581074:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314329.633970:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314330.592886:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314330.656829:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314331.616794:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314331.686359:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761314332.385430:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314332.385440:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761314332.385444:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761314332.385457:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2575 00000100:00080000:2.0:1761314332.385461:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761314332.385467:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761314332.385479:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761314332.385497:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000011b5c99f x1846869307493312/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761314332.385522:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314332.385526:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761314332.385704:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e1be9a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307493120:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314332.385821:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b3aaddc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307493248:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314332.385857:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761314332.385862:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008a641e2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307493184:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761314332.386207:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000011b5c99f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307493312:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00080000:1.0:1761314332.396793:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:3.0:1761314332.396804:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008a641e2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307493184:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761314332.396807:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761314332.396814:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000011b5c99f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307493312:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761314332.396826:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e1be9a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307493120:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314332.396842:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b3aaddc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307493248:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314332.640795:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314332.705403:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314333.665161:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314333.728925:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314334.688902:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314334.754066:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314335.712736:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314335.776876:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314336.739276:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314336.801027:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761314337.504814:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314337.504821:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761314337.504824:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761314337.504846:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2580 00000100:00080000:2.0:1761314337.504850:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761314337.504856:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761314337.504866:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761314337.504886:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000e1be9a7 x1846869307493568/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1761314337.504889:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ffb03e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307493376:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761314337.504925:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314337.504929:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761314337.504972:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e662bceb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307493440:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761314337.504999:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e1be9a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307493568:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761314337.505025:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b3aaddc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307493504:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314337.506980:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b3aaddc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307493504:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314337.507581:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e662bceb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307493440:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314337.511386:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ffb03e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307493376:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761314337.511994:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761314337.512028:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761314337.512035:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e1be9a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307493568:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761314337.760869:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314337.825271:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314338.784779:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314338.849257:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314339.808819:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314339.875522:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314340.840386:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314340.896916:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314341.857675:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314341.925187:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761314342.626461:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314342.626472:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761314342.626477:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761314342.626492:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2585 00000100:00080000:3.0:1761314342.626496:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761314342.626504:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761314342.626519:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761314342.626539:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000626f98db x1846869307493824/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761314342.626620:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000626f98db pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307493824:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761314342.626804:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000038534f0b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307493632:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314342.626842:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307493760:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314342.626872:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761314342.626877:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f133bb2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307493696:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761314342.627233:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314342.627239:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00080000:1.0:1761314342.634407:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761314342.634424:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761314342.634438:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000626f98db pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307493824:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761314342.635453:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000038534f0b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307493632:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314342.635473:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307493760:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314342.635495:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f133bb2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307493696:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314342.881014:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314342.947434:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314343.904795:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314343.969692:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314344.928818:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314344.993616:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314345.953250:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314346.017708:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314346.976839:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314347.043234:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761314347.744853:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314347.744860:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761314347.744863:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761314347.744888:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2590 00000100:00080000:3.0:1761314347.744891:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761314347.744898:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761314347.744908:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761314347.744924:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006e93398e x1846869307494080/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761314347.745015:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307494080:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00080000:3.0:1761314347.745366:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314347.745370:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761314347.745460:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000101f852a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307493888:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314347.747369:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307494016:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314347.747766:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761314347.747774:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307493952:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761314347.751635:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761314347.751667:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761314347.751677:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307494080:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761314347.757437:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307493952:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314347.757455:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307494016:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314347.759691:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000101f852a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307493888:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314348.000771:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314348.064996:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314349.024798:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314349.089815:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314350.048880:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314350.116302:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314351.074745:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314351.137722:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314352.097461:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314352.166660:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761314352.864877:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314352.864887:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761314352.864892:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761314352.864938:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2595 00000100:00080000:3.0:1761314352.864944:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761314352.864952:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761314352.864967:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761314352.864974:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f597cda6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307494144:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314352.864995:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000cc6dd8ca x1846869307494336/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761314352.865090:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cc6dd8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307494336:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761314352.865094:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307494272:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314352.865160:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761314352.865169:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307494208:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761314352.866731:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314352.866741:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761314352.874662:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307494208:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314352.874682:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f597cda6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307494144:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314352.874693:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307494272:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761314352.874833:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761314352.874838:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761314352.874843:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc6dd8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307494336:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761314353.122557:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314353.185069:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314354.144757:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314354.209527:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314355.168772:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314355.233577:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314356.193027:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314356.262160:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314357.216817:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314357.280844:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761314357.986036:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314357.986043:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761314357.986047:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761314357.986057:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2600 00000100:00080000:2.0:1761314357.986060:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00100000:3.0:1761314357.986060:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e662bceb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307494400:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761314357.986066:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761314357.986075:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761314357.986104:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ee08055d x1846869307494592/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761314357.986132:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314357.986135:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761314357.986148:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000011b5c99f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307494528:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314357.986157:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008a641e2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307494464:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761314357.986450:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ee08055d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307494592:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00080000:0.0:1761314357.993381:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761314357.993397:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761314357.993410:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ee08055d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307494592:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314357.993558:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008a641e2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307494464:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314357.993700:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000011b5c99f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307494528:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314357.999461:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e662bceb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307494400:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314358.242293:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314358.304985:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314359.264812:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314359.329908:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314360.289505:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314360.354755:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314361.313128:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314361.377184:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314362.340737:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314362.400912:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761314363.109934:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314363.109943:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761314363.109947:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761314363.109961:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2606 00000100:00080000:0.0:1761314363.109965:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761314363.109972:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761314363.109984:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761314363.110003:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000005a420586 x1846869307494848/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761314363.110034:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314363.110037:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761314363.110077:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307494720:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761314363.110180:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1761314363.110186:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307494656:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761314363.110213:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307494784:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314363.110536:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307494848:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761314363.117260:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307494656:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761314363.117287:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307494784:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761314363.117299:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307494720:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761314363.117573:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761314363.117579:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761314363.117584:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307494848:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761314363.360803:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314363.424865:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314364.388076:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314364.450971:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314365.408785:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314365.474864:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314366.432827:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314366.496911:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314367.461009:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314367.520873:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761314368.224891:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314368.224907:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761314368.224911:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761314368.224920:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2611 00000100:00080000:2.0:1761314368.224923:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761314368.224931:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761314368.224940:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761314368.224957:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000015f165e6 x1846869307495104/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1761314368.224977:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008a641e2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307494912:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761314368.224980:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314368.224991:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761314368.225047:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eca34e1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307494976:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761314368.225804:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000015f165e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307495104:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761314368.226887:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e3466f5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307495040:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314368.232325:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008a641e2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307494912:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314368.237601:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eca34e1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307494976:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761314368.242686:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761314368.242719:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761314368.242731:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000015f165e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307495104:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761314368.242897:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e3466f5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307495040:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314368.480921:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314368.546133:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314369.505187:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314369.569094:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314370.528822:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314370.593570:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314371.556484:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314371.616875:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314372.576794:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314372.641022:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761314373.346507:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314373.346514:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761314373.346517:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761314373.346527:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2616 00000100:00080000:3.0:1761314373.346530:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761314373.346535:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761314373.346557:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761314373.346574:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000041c8b6cb x1846869307495360/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761314373.346601:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314373.346604:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761314373.346624:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f364ec74 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307495232:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314373.346730:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1761314373.346735:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cc6dd8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307495168:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314373.346766:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007673f690 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307495296:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314373.346874:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000041c8b6cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307495360:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761314373.353766:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f364ec74 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307495232:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314373.354092:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007673f690 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307495296:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314373.354408:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc6dd8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307495168:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761314373.354674:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761314373.354679:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761314373.354683:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000041c8b6cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307495360:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761314373.600889:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314373.664893:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314374.624796:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314374.689913:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314375.648777:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314375.712884:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314376.672916:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314376.736870:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314377.698686:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314377.764275:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761314378.465163:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314378.465171:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761314378.465191:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761314378.465201:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2621 00000100:00080000:3.0:1761314378.465208:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761314378.465214:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761314378.465224:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761314378.465241:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e6d9e11e x1846869307495616/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761314378.465406:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307495744:192.168.203.151@tcp:17:kworker/1:1.0 00000100:00100000:1.0:1761314378.465488:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761314378.465494:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307495680:192.168.203.151@tcp:17:kworker/1:1.0 00000100:00100000:2.0:1761314378.465960:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cc6dd8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307495424:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314378.466015:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f364ec74 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307495552:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314378.466051:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761314378.466058:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007673f690 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307495488:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761314378.466186:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314378.466191:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761314378.466307:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307495616:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761314378.472739:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307495680:192.168.203.151@tcp:17:kworker/1:1.0 00000100:00100000:0.0:1761314378.472757:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307495744:192.168.203.151@tcp:17:kworker/1:1.0 00000100:00080000:0.0:1761314378.473278:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761314378.473285:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761314378.473290:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307495616:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761314378.473380:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007673f690 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307495488:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314378.473390:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc6dd8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307495424:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314378.473397:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f364ec74 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307495552:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314378.720919:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314378.785196:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314379.744804:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314379.811618:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314380.768936:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314380.836179:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314381.792847:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314381.859513:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314382.816744:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314382.882153:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761314383.586421:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307495808:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761314383.593789:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314383.593795:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761314383.593798:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761314383.593807:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2626 00000100:00080000:1.0:1761314383.593810:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761314383.593815:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761314383.593824:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761314383.593838:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000eb8b1f19 x1846869307496000/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761314383.593855:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314383.593857:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761314383.593915:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307495872:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314383.593988:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761314383.593994:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307495936:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314383.594748:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307496000:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761314383.599189:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307495808:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761314383.599347:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761314383.599352:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761314383.599356:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307496000:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761314383.607085:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307495936:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314383.607100:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307495872:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314383.841022:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314383.905193:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314384.868146:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314384.928909:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314385.895829:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314385.953825:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314386.912752:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314386.976928:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314387.936814:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314388.000850:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761314388.706159:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314388.706166:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761314388.706169:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761314388.706179:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2631 00000100:00080000:3.0:1761314388.706182:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761314388.706188:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761314388.706198:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761314388.706213:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f364ec74 x1846869307496256/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761314388.706268:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f364ec74 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307496256:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761314388.706624:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d32bed84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307496064:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314388.709224:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000009911ad3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307496192:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761314388.712099:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314388.712109:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761314388.712181:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e58cc722 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307496128:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761314388.718697:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761314388.718708:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761314388.718717:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f364ec74 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307496256:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761314388.724849:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000009911ad3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307496192:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314388.734708:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e58cc722 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307496128:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314388.735057:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d32bed84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307496064:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314388.960797:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314389.025600:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314389.984764:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314390.053200:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314391.008794:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314391.077187:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314392.032773:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314392.096880:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314393.056761:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314393.122516:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761314393.824972:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314393.824979:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761314393.824983:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761314393.824994:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2636 00000100:00080000:1.0:1761314393.824998:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761314393.825003:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761314393.825012:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761314393.825027:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000163e3801 x1846869307496512/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761314393.825048:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314393.825051:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761314393.825125:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307496512:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761314393.827209:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307496320:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761314393.827297:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307496448:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761314393.827328:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761314393.827333:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307496384:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761314393.839082:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761314393.839096:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761314393.839105:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307496512:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761314393.856129:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307496448:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761314393.857580:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307496384:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761314393.858832:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307496320:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314394.080769:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314394.144838:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314395.104766:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314395.169030:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314396.128757:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314396.195514:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314397.154712:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314397.218280:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314398.176977:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314398.241271:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761314398.946058:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314398.946068:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761314398.946073:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761314398.946086:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2641 00000100:00100000:2.0:1761314398.946087:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e58cc722 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307496576:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761314398.946090:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761314398.946096:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761314398.946108:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761314398.946124:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e6d9e11e x1846869307496768/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761314398.946150:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314398.946154:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761314398.946191:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007673f690 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307496704:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314398.946207:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cc6dd8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307496640:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314398.946341:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307496768:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761314398.957151:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc6dd8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307496640:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314398.957467:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e58cc722 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307496576:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314398.957484:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007673f690 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307496704:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761314398.960932:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761314398.960949:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761314398.960961:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307496768:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761314399.201221:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314399.267190:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314400.224804:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314400.289224:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314401.248884:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314401.312856:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314402.274113:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314402.337107:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314403.298092:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314403.364027:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761314404.064909:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314404.064916:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761314404.064920:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761314404.064930:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2647 00000100:00080000:1.0:1761314404.064934:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761314404.064941:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761314404.064951:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761314404.064971:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000163e3801 x1846869307497024/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761314404.064975:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307496832:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761314404.065016:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314404.065020:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761314404.065049:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307496896:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761314404.065109:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307496960:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314404.065361:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307497024:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00080000:2.0:1761314404.074307:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761314404.074327:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761314404.074336:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307497024:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761314404.074361:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307496896:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761314404.074446:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307496832:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761314404.074461:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307496960:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314404.320776:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314404.385337:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314405.345998:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314405.413977:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314406.368758:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314406.434615:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314407.393385:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314407.457539:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314408.416775:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314408.482333:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314409.186367:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cc6dd8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307497088:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761314409.187287:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1761314409.187291:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000041c8b6cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307497152:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761314409.187293:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761314409.187296:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761314409.187301:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2652 00000100:00080000:3.0:1761314409.187304:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761314409.187309:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761314409.187319:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761314409.187332:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f597cda6 x1846869307497280/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761314409.187352:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314409.187355:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761314409.187382:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f597cda6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307497280:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761314409.187385:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307497216:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314409.190824:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc6dd8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307497088:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761314409.192453:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761314409.192461:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761314409.192469:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f597cda6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307497280:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761314409.193530:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000041c8b6cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307497152:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314409.193675:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307497216:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314409.440771:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314409.506568:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314410.464766:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314410.529434:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314411.488853:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314411.553627:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314412.514337:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314412.578796:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314413.536754:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314413.600850:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761314414.306861:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314414.306869:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761314414.306872:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761314414.306883:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2657 00000100:00100000:1.0:1761314414.306886:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307497344:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761314414.306887:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761314414.306895:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761314414.306907:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761314414.306925:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007a68d9a8 x1846869307497536/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761314414.306952:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314414.306965:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761314414.306971:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307497472:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314414.306980:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307497536:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761314414.306986:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307497408:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761314414.310373:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307497344:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761314414.310395:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307497472:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761314414.310498:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307497408:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761314414.311396:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761314414.311402:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761314414.311407:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307497536:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761314414.560826:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314414.625190:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314415.584764:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314415.648829:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314416.609402:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314416.673057:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314417.633277:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314417.697503:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314418.656800:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314418.721041:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761314419.424899:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314419.424906:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761314419.424909:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761314419.424920:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2662 00000100:00080000:3.0:1761314419.424923:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761314419.424929:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761314419.424938:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761314419.424953:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000101f852a x1846869307497792/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761314419.424982:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314419.424985:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761314419.425024:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cc6dd8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307497664:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314419.425112:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1761314419.425118:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307497600:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314419.425138:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307497728:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314419.425261:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000101f852a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307497792:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761314419.429121:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307497728:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314419.429147:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc6dd8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307497664:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761314419.429696:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761314419.429704:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761314419.429710:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000101f852a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307497792:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761314419.432348:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307497600:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314419.680800:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314419.744858:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314420.704753:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314420.772622:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314421.728808:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314421.793169:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314422.752841:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314422.817199:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314423.777003:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314423.840920:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761314424.544886:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314424.544896:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761314424.544901:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761314424.544915:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2667 00000100:00080000:3.0:1761314424.544919:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761314424.544927:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761314424.544939:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761314424.544974:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a183f7ef x1846869307498048/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761314424.545049:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307498048:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761314424.545493:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307497856:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314424.545546:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307497984:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314424.545582:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761314424.545587:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cc6dd8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307497920:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761314424.546195:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314424.546201:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00080000:1.0:1761314424.555275:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761314424.555289:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761314424.555300:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307498048:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761314424.555413:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc6dd8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307497920:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314424.555452:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307497984:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314424.558863:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307497856:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314424.803590:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314424.865397:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314425.824840:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314425.889010:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314426.848792:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314426.912885:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314427.876161:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314427.937108:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314428.897592:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314428.962342:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761314429.665248:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314429.665258:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761314429.665263:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761314429.665278:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2672 00000100:00080000:3.0:1761314429.665282:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00100000:2.0:1761314429.665289:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307498112:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761314429.665291:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761314429.665317:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761314429.665339:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000d82b82f x1846869307498304/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761314429.665376:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314429.665380:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761314429.665427:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307498176:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314429.665438:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f133bb2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307498240:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314429.665569:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307498304:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00080000:1.0:1761314429.673365:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761314429.673380:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761314429.673392:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307498304:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761314429.674415:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307498112:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314429.679156:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307498176:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314429.679736:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f133bb2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307498240:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314429.920782:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314429.986966:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314430.944811:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314431.009801:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314431.968754:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314432.034883:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314432.992797:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314433.058164:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314434.016797:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314434.080887:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761314434.784901:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314434.784913:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761314434.784917:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761314434.784932:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2677 00000100:00080000:3.0:1761314434.784937:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 6s 00000100:00080000:3.0:1761314434.784945:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761314434.784958:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761314434.784977:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006c8fcfe5 x1846869307498560/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761314434.785005:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314434.785008:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761314434.785113:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307498560:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761314434.785821:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000038534f0b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307498432:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314434.785890:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1761314434.785898:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307498368:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314434.785924:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000626f98db pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307498496:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314434.795752:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307498368:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314434.795783:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000626f98db pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307498496:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314434.795796:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000038534f0b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307498432:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761314434.796258:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761314434.796279:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761314434.796285:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307498560:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761314435.040907:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314435.107974:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314436.064802:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314436.128976:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314437.089025:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314437.152907:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314438.112819:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314438.177460:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314439.137362:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314439.200877:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761314439.909185:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314439.909195:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761314439.909199:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761314439.909213:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2682 00000100:00080000:1.0:1761314439.909219:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 11s 00000100:00080000:1.0:1761314439.909226:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761314439.909238:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761314439.909258:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000eb8b1f19 x1846869307498816/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761314439.909345:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307498816:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00080000:1.0:1761314439.909885:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314439.909890:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761314439.910250:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307498624:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314439.910314:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307498752:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314439.910342:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761314439.910348:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307498688:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314439.917905:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307498688:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314439.917932:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307498752:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761314439.918039:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761314439.918044:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761314439.918051:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307498816:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761314439.920251:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307498624:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314440.160922:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314440.228407:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314441.184760:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314441.249276:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314442.208787:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314442.272876:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314443.232723:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314443.297040:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314444.256770:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314444.320866:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761314445.024851:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314445.024856:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761314445.024858:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761314445.024866:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2687 00000100:00080000:2.0:1761314445.024871:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 16s 00000100:00080000:2.0:1761314445.024876:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761314445.024884:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761314445.024896:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e9376149 x1846869307499072/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761314445.024917:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314445.024919:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761314445.024937:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eca34e1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307498880:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314445.024947:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002292ab57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307498944:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761314445.024959:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e9376149 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307499072:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761314445.025722:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ba5c8e77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307499008:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:02020000:1.0:1761314445.030004:0:2262:0:(client.c:1361:ptlrpc_check_status()) 11-0: lustre-OST0000-osc-ffff951bc4d8c800: operation ost_connect to node 192.168.203.151@tcp failed: rc = -19 00000100:00100000:3.0:1761314445.030160:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eca34e1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307498880:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314445.030179:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ba5c8e77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307499008:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314445.030284:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002292ab57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307498944:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761314445.037435:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761314445.037463:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761314445.037473:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e9376149 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307499072:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761314445.280836:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314445.344851:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314446.304781:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314446.368855:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314447.329424:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314447.392847:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314448.352864:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314448.417598:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314449.376763:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314449.442096:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761314450.144903:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314450.144911:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761314450.144914:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761314450.144925:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2693 00000100:00080000:3.0:1761314450.144929:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761314450.144935:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761314450.144958:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761314450.144973:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ddd3c50a x1846869307499328/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761314450.144998:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314450.145001:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761314450.145021:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cdcc674d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307499200:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761314450.145055:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307499328:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314450.145122:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307499136:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314450.145239:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307499264:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761314450.149916:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761314450.149928:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761314450.149937:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307499328:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314450.150004:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307499136:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314450.150017:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307499264:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314450.150095:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cdcc674d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307499200:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314450.400934:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314450.465430:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314451.424750:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314451.489104:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314452.449269:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314452.514815:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314453.472815:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314453.537408:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314454.496828:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314454.561285:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761314455.264868:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314455.264875:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761314455.264878:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761314455.264901:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2698 00000100:00080000:2.0:1761314455.264904:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761314455.264910:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761314455.264919:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761314455.264933:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000039978a4e x1846869307499584/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761314455.264958:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314455.264961:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761314455.264997:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b2298797 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307499456:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761314455.265012:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039978a4e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307499584:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761314455.265282:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002292ab57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307499392:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314455.265593:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ae09c97d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307499520:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761314455.274487:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761314455.274498:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761314455.274507:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039978a4e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307499584:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761314455.275363:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002292ab57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307499392:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314455.275378:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ae09c97d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307499520:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314455.275698:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b2298797 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307499456:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314455.521518:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314455.584846:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314456.546373:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314456.613105:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314457.568806:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314457.632891:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314458.593333:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314458.658469:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314459.616762:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314459.682880:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761314460.388237:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314460.388244:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761314460.388247:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761314460.388257:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2703 00000100:00080000:3.0:1761314460.388260:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761314460.388265:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000100:00100000:2.0:1761314460.388266:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cdcc674d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307499648:192.168.203.151@tcp:400:kworker/u8:1.0 00000008:00100000:3.0:1761314460.388273:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761314460.388287:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000004505177d x1846869307499840/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761314460.388306:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314460.388309:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761314460.388332:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307499712:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314460.388361:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004e00d332 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307499776:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761314460.388900:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307499840:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314460.397356:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cdcc674d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307499648:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314460.397381:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004e00d332 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307499776:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314460.397550:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307499712:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761314460.402977:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761314460.402990:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761314460.403002:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307499840:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761314460.640889:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314460.705769:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314461.665704:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314461.737864:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314462.693560:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314462.752920:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314463.716246:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314463.776889:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314464.736762:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314464.800797:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761314465.504937:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314465.504944:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761314465.504947:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761314465.504956:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2708 00000100:00080000:2.0:1761314465.504972:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761314465.504978:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761314465.504987:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761314465.505002:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f220504f x1846869307500096/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761314465.505062:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f220504f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307500096:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761314465.505195:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b2298797 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307499904:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314465.505227:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aea3302e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307500032:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314465.505266:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761314465.505272:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001426be00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307499968:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761314465.506751:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314465.506760:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761314465.508802:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001426be00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307499968:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314465.508820:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b2298797 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307499904:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314465.508829:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aea3302e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307500032:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761314465.509409:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761314465.509414:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761314465.509420:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f220504f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307500096:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761314465.760989:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314465.824906:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314466.784994:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314466.850546:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314467.808729:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314467.874186:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314468.832763:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314468.898663:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314469.856765:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314469.920868:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761314470.624882:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314470.624889:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761314470.624892:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761314470.624902:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2713 00000100:00080000:3.0:1761314470.624906:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761314470.624911:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761314470.624920:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761314470.624934:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006c8fcfe5 x1846869307500352/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761314470.624979:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314470.624983:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761314470.625006:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307500352:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761314470.625006:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307500224:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314470.625176:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1761314470.625183:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307500160:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314470.625214:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307500288:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314470.628726:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307500288:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314470.628741:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307500224:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314470.634446:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307500160:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761314470.636026:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761314470.636036:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761314470.636046:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307500352:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314470.880964:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314470.946574:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314471.905275:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314471.969374:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314472.929248:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314472.994922:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314473.952795:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314474.016907:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314474.976861:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314475.040962:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761314475.744839:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314475.744847:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761314475.744850:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761314475.744861:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2718 00000100:00080000:3.0:1761314475.744864:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761314475.744870:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761314475.744880:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761314475.744895:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000038534f0b x1846869307500608/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761314475.744923:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314475.744926:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761314475.744951:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307500480:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314475.745076:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1761314475.745082:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307500416:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314475.745110:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307500544:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761314475.745278:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000038534f0b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307500608:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761314475.749290:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307500480:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314475.755733:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307500416:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314475.762391:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307500544:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761314475.764490:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761314475.764503:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761314475.764513:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000038534f0b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307500608:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314476.000794:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314476.065119:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314477.027290:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314477.097903:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314478.048870:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314478.119263:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314479.072739:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314479.139503:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314480.096757:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314480.160875:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761314480.865310:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307500672:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314480.872065:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307500672:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761314480.872346:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314480.872355:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761314480.872359:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761314480.872366:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2723 00000100:00080000:1.0:1761314480.872369:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1761314480.872377:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761314480.872390:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761314480.872412:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000540c3ce6 x1846869307500864/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761314480.872492:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307500864:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00080000:1.0:1761314480.874826:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314480.874838:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761314480.875028:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307500736:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314480.875132:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761314480.875138:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307500800:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761314480.891413:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761314480.891426:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761314480.891437:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307500864:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761314480.891679:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307500800:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314480.891712:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307500736:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314481.120813:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314481.184902:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314482.144802:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314482.208866:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314483.168783:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314483.233492:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314484.193699:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314484.257469:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314485.216813:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314485.283139:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761314485.984932:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314485.984940:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761314485.984944:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761314485.984965:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2728 00000100:00080000:3.0:1761314485.984970:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761314485.984977:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761314485.984988:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761314485.985006:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000626f98db x1846869307501120/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761314485.985061:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000626f98db pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307501120:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314485.985214:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307500928:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314485.985251:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307501056:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314485.985283:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761314485.985288:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307500992:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761314485.985471:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314485.985476:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00080000:1.0:1761314485.993753:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761314485.993783:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761314485.993814:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000626f98db pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307501120:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314485.994294:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307500992:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314485.994336:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307500928:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314485.994349:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307501056:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314486.242533:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314486.306104:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314487.264960:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314487.330541:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314488.289174:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314488.352835:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314489.312810:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314489.376974:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314490.336844:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314490.403386:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761314491.106082:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314491.106091:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761314491.106095:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761314491.106108:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2734 00000100:00080000:3.0:1761314491.106129:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761314491.106138:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761314491.106150:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761314491.106170:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000d82b82f x1846869307501376/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761314491.106195:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314491.106199:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761314491.106477:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307501376:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314491.107427:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307501184:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314491.108329:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307501312:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314491.108402:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761314491.108410:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f133bb2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307501248:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314491.114750:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307501184:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761314491.114948:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761314491.114955:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761314491.114960:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307501376:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314491.118415:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307501312:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314491.119453:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f133bb2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307501248:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314491.360969:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314491.424888:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314492.384742:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314492.449324:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314493.408784:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314493.472855:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314494.432967:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314494.497796:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314495.459614:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314495.520817:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761314496.224864:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314496.224871:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761314496.224875:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761314496.224915:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2739 00000100:00080000:3.0:1761314496.224918:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761314496.224924:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761314496.224933:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761314496.224948:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a183f7ef x1846869307501632/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761314496.224966:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307501440:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761314496.224973:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314496.224977:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761314496.225026:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307501504:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314496.225069:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cc6dd8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307501568:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761314496.225892:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307501632:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314496.230121:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307501440:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314496.232596:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307501504:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314496.232774:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc6dd8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307501568:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761314496.235831:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761314496.235842:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761314496.235852:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307501632:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314496.480767:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314496.545668:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314497.504740:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314497.568835:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314498.529667:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314498.593552:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314499.553463:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314499.619954:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314500.577125:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314500.640909:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761314501.344805:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314501.344813:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761314501.344816:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761314501.344826:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2744 00000100:00080000:2.0:1761314501.344830:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761314501.344837:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761314501.344874:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761314501.344891:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000004592c0a5 x1846869307501888/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761314501.344976:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004592c0a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307501888:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00080000:2.0:1761314501.345070:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314501.345074:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761314501.345096:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b4eac89a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307501760:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314501.345131:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1761314501.345136:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000039eebc2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307501696:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314501.345157:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307501824:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314501.352033:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000039eebc2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307501696:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761314501.354609:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:2.0:1761314501.354635:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b4eac89a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307501760:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761314501.354657:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761314501.354662:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004592c0a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307501888:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761314501.354759:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307501824:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314501.601032:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314501.664992:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314502.626521:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314502.690239:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314503.648701:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314503.712879:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314504.672768:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314504.736883:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314505.699113:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314505.764018:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314506.464905:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307501952:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761314506.466388:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314506.466394:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761314506.466397:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761314506.466404:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2749 00000100:00080000:2.0:1761314506.466407:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761314506.466413:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761314506.466421:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761314506.466446:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000007136630 x1846869307502144/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761314506.466509:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000007136630 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307502144:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761314506.467125:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000039eebc2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307502080:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761314506.467255:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314506.467259:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761314506.467279:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b4eac89a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307502016:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314506.472360:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000039eebc2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307502080:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314506.472383:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307501952:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314506.472492:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b4eac89a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307502016:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761314506.472786:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761314506.472791:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761314506.472796:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000007136630 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307502144:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314506.721400:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314506.785906:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314507.745969:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314507.809358:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314508.768827:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314508.833508:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314509.792969:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314509.856908:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314510.817402:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314510.882115:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761314511.592265:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314511.592275:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761314511.592279:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761314511.592292:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2754 00000100:00080000:3.0:1761314511.592297:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761314511.592304:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761314511.592316:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761314511.592335:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f597cda6 x1846869307502400/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761314511.592364:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314511.592368:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761314511.592400:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000101f852a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307502272:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314511.592514:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1761314511.592522:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307502208:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314511.592568:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000041c8b6cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307502336:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761314511.592890:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f597cda6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307502400:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761314511.598584:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000041c8b6cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307502336:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314511.598606:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000101f852a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307502272:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761314511.602968:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761314511.602983:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761314511.602995:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f597cda6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307502400:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761314511.605808:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307502208:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314511.840842:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314511.904857:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314512.866725:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314512.929741:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314513.888719:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314513.952887:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314514.914562:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314514.977397:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314515.936807:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314516.000904:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761314516.704910:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314516.704919:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761314516.704923:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761314516.704937:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2759 00000100:00080000:0.0:1761314516.704941:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761314516.704948:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761314516.704960:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761314516.704980:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008e59d025 x1846869307502656/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761314516.705030:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314516.705034:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761314516.705066:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307502528:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314516.705182:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1761314516.705189:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307502464:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314516.705214:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307502592:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314516.705577:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307502656:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761314516.712306:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307502464:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761314516.716551:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761314516.716566:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761314516.716577:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307502656:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761314516.717256:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307502592:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314516.717274:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307502528:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314516.960727:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314517.024927:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314517.986075:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314518.050885:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314519.009607:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314519.073025:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314520.033568:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314520.097679:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314521.058369:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314521.120868:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761314521.824910:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307502720:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314521.825118:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761314521.825126:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307502784:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761314521.827781:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314521.827788:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761314521.827791:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761314521.827801:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2764 00000100:00080000:0.0:1761314521.827804:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761314521.827809:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761314521.827819:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761314521.827834:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ace4d025 x1846869307502912/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761314521.827874:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314521.827878:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761314521.827911:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307502912:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761314521.827933:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761314521.831722:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307502848:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314521.831964:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307502784:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314521.831976:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307502720:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761314521.838771:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761314521.838785:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761314521.838798:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307502912:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761314521.839840:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307502848:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314522.080815:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314522.146463:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314523.104834:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314523.168849:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314524.128819:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314524.194929:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314525.152779:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314525.218481:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314526.178091:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314526.244268:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761314526.945011:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314526.945038:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761314526.945042:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761314526.945054:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2769 00000100:00080000:2.0:1761314526.945058:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761314526.945065:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761314526.945076:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761314526.945093:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000039eebc2 x1846869307503168/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761314526.945117:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314526.945121:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761314526.945144:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000007136630 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307503040:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314526.945350:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b4eac89a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307502976:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761314526.946076:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000039eebc2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307503168:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314526.946778:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307503104:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761314526.955443:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761314526.955457:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761314526.955469:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000039eebc2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307503168:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314526.955572:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b4eac89a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307502976:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314526.955589:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307503104:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314526.956872:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000007136630 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307503040:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314527.201442:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314527.264948:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314528.224948:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314528.291769:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314529.248799:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314529.312906:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314530.273461:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314530.338263:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314531.299557:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314531.360852:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761314532.066034:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314532.066043:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761314532.066047:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761314532.066062:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2775 00000100:00080000:3.0:1761314532.066067:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761314532.066074:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761314532.066086:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761314532.066103:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e6d9e11e x1846869307503424/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761314532.066130:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314532.066133:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761314532.066180:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307503424:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314532.067758:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307503232:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314532.068075:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000041c8b6cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307503360:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314532.068115:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761314532.068143:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000101f852a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307503296:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761314532.073028:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761314532.073044:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761314532.073056:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307503424:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314532.077067:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000101f852a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307503296:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314532.077089:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307503232:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314532.077099:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000041c8b6cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307503360:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314532.321319:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314532.386284:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314533.349996:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314533.408919:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314534.369305:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314534.437712:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314535.393539:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314535.456953:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314536.416794:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314536.480986:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761314537.185263:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314537.185278:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761314537.185282:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761314537.185298:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2780 00000100:00080000:1.0:1761314537.185303:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1761314537.185312:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761314537.185326:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761314537.185348:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a1719d14 x1846869307503680/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761314537.185430:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307503680:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761314537.186171:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307503488:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314537.186214:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307503616:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314537.186240:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761314537.186246:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307503552:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761314537.188151:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314537.188189:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761314537.202787:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307503616:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314537.202820:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307503552:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314537.203360:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307503488:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761314537.206915:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761314537.206925:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761314537.206935:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307503680:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761314537.441292:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314537.509241:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314538.464775:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314538.529226:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314539.488759:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314539.555128:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314540.512752:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314540.585813:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314541.536798:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314541.606009:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761314542.309056:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314542.309064:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761314542.309067:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761314542.309078:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2785 00000100:00080000:2.0:1761314542.309081:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761314542.309086:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761314542.309110:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761314542.309127:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f220504f x1846869307503936/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761314542.309152:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314542.309155:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761314542.309183:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b4eac89a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307503808:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314542.309277:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1761314542.309283:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307503744:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314542.309304:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004592c0a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307503872:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761314542.309959:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f220504f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307503936:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314542.318236:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307503744:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314542.318259:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004592c0a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307503872:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314542.318270:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b4eac89a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307503808:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761314542.321749:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761314542.321760:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761314542.321770:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f220504f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307503936:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314542.560771:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314542.626259:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314543.585033:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314543.648860:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314544.609778:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314544.676762:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314545.633353:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314545.696883:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314546.656776:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314546.720882:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761314547.426122:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314547.426128:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761314547.426131:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761314547.426140:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2790 00000100:00080000:3.0:1761314547.426143:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761314547.426149:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761314547.426157:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761314547.426170:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000009911ad3 x1846869307504192/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761314547.426210:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314547.426213:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761314547.426237:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e58cc722 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307504064:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761314547.426265:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000009911ad3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307504192:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314547.427449:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007673f690 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307504000:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314547.429251:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d32bed84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307504128:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314547.438454:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e58cc722 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307504064:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761314547.439051:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761314547.439056:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761314547.439062:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000009911ad3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307504192:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314547.439413:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007673f690 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307504000:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314547.441546:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d32bed84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307504128:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314547.680801:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314547.744907:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314548.704715:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314548.769146:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314549.729231:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314549.792931:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314550.752720:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314550.818192:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314551.777346:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314551.841954:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761314552.545749:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314552.545759:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761314552.545763:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761314552.545790:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2795 00000100:00080000:0.0:1761314552.545806:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761314552.545814:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761314552.545851:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761314552.545897:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000bcaee194 x1846869307504448/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761314552.545931:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314552.545934:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761314552.545985:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307504320:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314552.546067:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307504448:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761314552.546445:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307504256:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314552.548407:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307504384:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314552.558189:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307504320:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314552.558257:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307504256:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314552.558272:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307504384:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761314552.562394:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761314552.562411:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761314552.562424:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307504448:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761314552.800812:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314552.865015:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314553.825255:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314553.893707:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314554.848836:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314554.913800:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314555.872897:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314555.937108:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314556.898248:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314556.964988:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761314557.664981:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314557.664990:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761314557.664994:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761314557.665006:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2800 00000100:00080000:3.0:1761314557.665010:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761314557.665016:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761314557.665027:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761314557.665043:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007673f690 x1846869307504704/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761314557.665065:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314557.665068:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761314557.665160:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007673f690 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307504704:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314557.665402:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f364ec74 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307504576:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314557.666577:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1761314557.666591:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e58cc722 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307504512:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314557.666635:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d32bed84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307504640:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314557.678767:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f364ec74 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307504576:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314557.680773:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e58cc722 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307504512:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314557.680797:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d32bed84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307504640:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761314557.685590:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761314557.685605:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761314557.685617:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007673f690 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307504704:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314557.921456:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314557.985278:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314558.944695:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314559.009035:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314559.968922:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314560.033226:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314560.993007:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314561.058341:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314562.017012:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314562.082477:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761314562.784930:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314562.784941:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761314562.784945:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761314562.784958:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2805 00000100:00080000:0.0:1761314562.784963:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761314562.784971:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761314562.784982:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761314562.785021:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000034a6cd79 x1846869307504960/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761314562.786664:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314562.786670:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761314562.786694:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307504960:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761314562.787503:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307504768:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314562.787607:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307504896:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314562.787685:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761314562.787693:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307504832:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761314562.792520:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761314562.792533:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761314562.792544:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307504960:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761314562.792932:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307504832:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314562.792945:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307504768:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314562.792957:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307504896:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314563.043683:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314563.104930:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314564.065173:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314564.131698:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314565.089861:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314565.154355:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314566.114826:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314566.176928:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314567.136794:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314567.200934:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761314567.904930:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314567.904939:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761314567.904943:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761314567.904957:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2810 00000100:00080000:0.0:1761314567.904961:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761314567.904968:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761314567.904981:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761314567.904998:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000094820351 x1846869307505216/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761314567.905024:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314567.905028:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761314567.905130:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307505024:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314567.905244:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307505152:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314567.905281:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761314567.905286:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307505088:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314567.905815:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307505216:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761314567.908177:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307505152:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314567.908218:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307505088:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314567.911428:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307505024:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761314567.912705:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761314567.912715:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761314567.912723:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307505216:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761314568.160897:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314568.225706:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314569.185228:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314569.250063:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314570.208764:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314570.281466:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314571.233211:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314571.296864:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314572.257184:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314572.321892:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761314573.025447:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314573.025453:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761314573.025455:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761314573.025466:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2815 00000100:00080000:3.0:1761314573.025468:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761314573.025473:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761314573.025481:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761314573.025494:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000041c8b6cb x1846869307505472/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761314573.025556:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000041c8b6cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307505472:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314573.027524:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d32bed84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307505280:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314573.027596:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000009911ad3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307505408:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314573.027621:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761314573.027676:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e58cc722 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307505344:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761314573.029063:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314573.029070:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00080000:1.0:1761314573.043426:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761314573.043437:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761314573.043447:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000041c8b6cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307505472:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314573.044534:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e58cc722 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307505344:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314573.044547:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d32bed84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307505280:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314573.049015:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000009911ad3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307505408:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314573.280768:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314573.344929:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314574.308491:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314574.368843:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314575.329009:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314575.393358:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314576.352878:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314576.425086:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314577.376810:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314577.444079:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761314578.145128:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307505536:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761314578.146245:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314578.146263:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761314578.146267:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761314578.146273:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2821 00000100:00080000:0.0:1761314578.146277:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761314578.146282:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761314578.146291:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761314578.146319:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000005b900d12 x1846869307505728/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761314578.146398:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307505728:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761314578.146413:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307505664:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314578.146880:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761314578.146888:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307505600:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761314578.147933:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314578.147940:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761314578.151393:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307505600:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314578.151412:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307505664:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314578.151421:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307505536:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761314578.155161:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761314578.155172:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761314578.155180:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307505728:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761314578.401770:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314578.466059:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314579.428510:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314579.488898:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314580.448774:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314580.512848:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314581.472771:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314581.536906:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314582.496769:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314582.560908:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761314583.265160:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314583.265166:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761314583.265168:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761314583.265177:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2826 00000100:00080000:1.0:1761314583.265181:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1761314583.265185:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761314583.265193:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761314583.265204:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000eb8b1f19 x1846869307505984/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761314583.265222:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314583.265225:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761314583.265301:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307505984:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761314583.265558:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307505856:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314583.265606:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1761314583.265613:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307505792:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314583.265635:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307505920:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761314583.268973:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761314583.268988:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761314583.269001:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307505984:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761314583.269017:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307505792:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314583.269062:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307505920:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314583.269072:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307505856:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314583.521602:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314583.587013:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314584.548146:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314584.612513:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314585.571775:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314585.633037:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314586.592809:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314586.657783:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314587.617308:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314587.680893:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761314588.384907:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314588.384918:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761314588.384922:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761314588.384934:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2831 00000100:00080000:3.0:1761314588.384938:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761314588.384944:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761314588.384976:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761314588.384996:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f597cda6 x1846869307506240/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761314588.385030:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314588.385034:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761314588.385202:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f597cda6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307506240:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314588.385468:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307506048:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314588.385504:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307506176:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314588.385528:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761314588.385533:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000101f852a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307506112:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314588.390478:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307506048:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314588.390717:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000101f852a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307506112:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314588.390738:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307506176:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761314588.391912:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761314588.391922:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761314588.391930:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f597cda6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307506240:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314588.640798:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314588.704999:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314589.664797:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314589.729841:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314590.689079:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314590.758171:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314591.714961:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314591.776783:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314592.736769:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314592.801244:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761314593.504949:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314593.504959:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761314593.504981:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761314593.504994:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2836 00000100:00080000:1.0:1761314593.504998:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761314593.505006:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761314593.505020:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761314593.505042:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000003871442b x1846869307506496/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761314593.505071:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314593.505075:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761314593.505124:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307506496:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761314593.505179:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307506304:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314593.505262:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307506432:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314593.505295:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761314593.505301:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307506368:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761314593.507790:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761314593.507800:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761314593.507809:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307506496:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761314593.513948:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307506368:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314593.513963:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307506304:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314593.513971:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307506432:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314593.761631:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314593.824940:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314594.784772:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314594.850706:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314595.812139:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314595.872880:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314596.832760:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314596.896821:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314597.856758:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314597.922279:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761314598.624964:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314598.624982:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761314598.624986:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761314598.624997:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2841 00000100:00080000:1.0:1761314598.625001:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761314598.625022:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761314598.625034:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761314598.625068:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a1719d14 x1846869307506752/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761314598.625165:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314598.625169:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761314598.625331:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307506560:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314598.625951:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307506688:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314598.626342:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307506752:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761314598.627524:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761314598.627555:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307506624:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:02020000:2.0:1761314598.633571:0:2262:0:(client.c:1361:ptlrpc_check_status()) 11-0: lustre-OST0000-osc-ffff951bc4d8c800: operation ost_connect to node 192.168.203.151@tcp failed: rc = -19 00000100:00080000:2.0:1761314598.633587:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761314598.633594:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761314598.633598:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307506752:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761314598.633896:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307506624:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314598.633909:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307506560:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314598.633920:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307506688:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314598.880791:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314598.944879:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314599.904823:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314599.970917:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314600.929217:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314600.994685:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314601.952817:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314602.019515:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314602.976932:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314603.042262:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761314603.745905:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314603.745915:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761314603.745919:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761314603.745933:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2846 00000100:00080000:0.0:1761314603.745937:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761314603.745944:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761314603.745958:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761314603.745977:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007dcceb4a x1846869307507008/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761314603.746003:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314603.746007:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761314603.746039:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307506880:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314603.746175:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1761314603.746181:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307506816:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314603.746225:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307506944:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314603.746547:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307507008:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761314603.757557:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307506880:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761314603.758959:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761314603.758968:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761314603.758978:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307507008:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761314603.761956:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307506816:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314603.761983:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307506944:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314604.000773:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314604.065899:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314605.026102:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314605.089816:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314606.049050:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314606.113073:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314607.072795:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314607.141036:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314608.098080:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314608.162356:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761314608.865376:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314608.865384:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761314608.865387:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761314608.865397:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2851 00000100:00080000:2.0:1761314608.865400:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761314608.865420:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761314608.865431:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761314608.865445:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000aea3302e x1846869307507264/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761314608.865472:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314608.865475:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761314608.865500:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004592c0a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307507136:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314608.865595:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1761314608.865602:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b4eac89a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307507072:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314608.865627:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307507200:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761314608.865829:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aea3302e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307507264:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314608.868632:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307507200:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314608.868674:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004592c0a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307507136:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314608.869024:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b4eac89a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307507072:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761314608.869101:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761314608.869105:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761314608.869121:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aea3302e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307507264:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314609.120765:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314609.185991:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314610.146471:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314610.208872:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314611.168758:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314611.232864:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314612.192763:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314612.258627:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314613.216998:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314613.284723:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761314613.984960:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314613.984971:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761314613.984975:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761314613.984989:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2856 00000100:00080000:0.0:1761314613.984994:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761314613.985001:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761314613.985016:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761314613.985034:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000057b18867 x1846869307507520/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761314613.985061:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314613.985066:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761314613.985090:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307507328:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314613.985216:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307507456:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314613.985255:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761314613.985261:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307507392:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314613.985466:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307507520:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761314613.990767:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307507328:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314613.993751:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307507392:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314613.993777:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307507456:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761314613.993828:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761314613.993835:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761314613.993840:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307507520:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761314614.242191:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314614.305396:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314615.264801:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314615.328890:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314616.288805:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314616.352882:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314617.312815:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314617.376940:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314618.336904:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314618.401208:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761314619.104857:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307507584:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761314619.107495:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314619.107502:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761314619.107505:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761314619.107515:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2862 00000100:00080000:0.0:1761314619.107518:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761314619.107524:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761314619.107533:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761314619.107549:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000057b18867 x1846869307507776/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761314619.107576:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314619.107579:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761314619.107610:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307507712:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314619.107698:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761314619.107705:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307507648:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314619.107913:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307507776:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761314619.113408:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307507712:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314619.113428:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307507584:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314619.113453:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307507648:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761314619.118577:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761314619.118587:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761314619.118597:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307507776:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761314619.360772:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314619.424822:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314620.384772:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314620.448955:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314621.408763:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314621.482157:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314622.432910:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314622.496922:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314623.457262:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314623.521498:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314624.225675:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b4eac89a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307507840:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314624.225863:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761314624.225870:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004592c0a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307507904:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761314624.226067:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314624.226072:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761314624.226075:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761314624.226080:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2867 00000100:00080000:2.0:1761314624.226083:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761314624.226089:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761314624.226099:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761314624.226113:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b2298797 x1846869307508032/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761314624.226140:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314624.226144:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761314624.226156:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761314624.226181:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b2298797 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307508032:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314624.226182:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307507968:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314624.231082:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307507968:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314624.231120:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004592c0a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307507904:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761314624.235206:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761314624.235216:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761314624.235225:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b2298797 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307508032:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761314624.235567:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b4eac89a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307507840:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314624.482689:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314624.545153:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314625.504763:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314625.568874:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314626.530841:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314626.594478:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314627.552767:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314627.620041:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314628.576787:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314628.641021:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314629.345340:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307508096:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314629.346590:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761314629.346603:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cc6dd8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307508160:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761314629.347175:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314629.347180:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761314629.347195:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761314629.347200:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2872 00000100:00080000:3.0:1761314629.347203:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761314629.347208:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761314629.347216:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761314629.347230:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f133bb2e x1846869307508288/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761314629.347250:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314629.347253:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761314629.347268:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761314629.347276:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307508224:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761314629.347352:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f133bb2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307508288:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314629.355421:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307508096:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314629.356774:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307508224:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314629.357610:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc6dd8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307508160:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761314629.357709:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761314629.357715:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761314629.357720:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f133bb2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307508288:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314629.600791:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314629.664821:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314630.625521:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314630.689444:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314631.648954:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314631.712892:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314632.672809:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314632.738020:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314633.696794:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314633.761217:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761314634.467327:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314634.467358:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761314634.467363:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761314634.467377:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2877 00000100:00080000:1.0:1761314634.467381:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761314634.467390:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761314634.467401:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761314634.467421:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a15ef099 x1846869307508544/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761314634.467506:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307508544:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761314634.467748:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307508352:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314634.467779:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307508480:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314634.467803:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761314634.467808:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307508416:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761314634.469107:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314634.469118:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761314634.471685:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307508416:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314634.471713:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307508352:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314634.471730:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307508480:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761314634.471797:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761314634.471803:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761314634.471809:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307508544:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761314634.721086:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314634.785293:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314635.744802:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314635.809493:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314636.770242:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314636.838880:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314637.792800:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314637.861324:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314638.816800:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314638.880910:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761314639.584892:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314639.584902:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761314639.584906:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761314639.584919:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2882 00000100:00080000:1.0:1761314639.584923:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761314639.584930:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761314639.584943:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761314639.584960:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000163e3801 x1846869307508800/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761314639.585013:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314639.585017:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761314639.585050:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307508672:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314639.585158:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1761314639.585165:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307508608:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314639.585189:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307508736:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314639.585558:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307508800:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761314639.587535:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307508608:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314639.588428:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307508736:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314639.588440:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307508672:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761314639.589511:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761314639.589519:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761314639.589525:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307508800:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761314639.842729:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314639.904915:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314640.865158:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314640.930119:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314641.889161:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314641.952987:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314642.912836:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314642.978819:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314643.937602:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314644.002289:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761314644.705043:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314644.705053:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761314644.705057:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761314644.705069:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2887 00000100:00080000:2.0:1761314644.705073:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761314644.705079:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761314644.705091:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761314644.705106:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000002292ab57 x1846869307509056/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761314644.705165:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002292ab57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307509056:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761314644.705308:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307508864:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314644.705346:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ae09c97d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307508992:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314644.705378:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761314644.705404:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001426be00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307508928:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761314644.705560:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314644.705565:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00080000:1.0:1761314644.711201:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761314644.711216:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761314644.711228:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002292ab57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307509056:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761314644.714381:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307508864:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314644.718572:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001426be00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307508928:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314644.718607:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ae09c97d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307508992:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314644.961356:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314645.026552:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314645.986100:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314646.050191:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314647.011735:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314647.073392:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314648.032876:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314648.097633:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314649.056910:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314649.122319:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761314649.828835:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314649.828842:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761314649.828844:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761314649.828854:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2892 00000100:00080000:0.0:1761314649.828857:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761314649.828863:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761314649.828886:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761314649.828903:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007dcceb4a x1846869307509312/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761314649.828933:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314649.828936:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761314649.828965:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307509184:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314649.829056:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1761314649.829063:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307509120:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314649.829085:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307509248:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314649.829433:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307509312:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761314649.831549:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307509120:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314649.839196:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307509248:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314649.839216:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307509184:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761314649.842500:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761314649.842510:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761314649.842519:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307509312:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761314650.080753:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314650.144904:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314651.104758:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314651.168878:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314652.129263:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314652.192886:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314653.152791:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314653.218693:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314654.176884:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314654.246232:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761314654.945546:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314654.945556:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761314654.945560:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761314654.945575:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2897 00000100:00100000:3.0:1761314654.945585:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039978a4e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307509376:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761314654.945598:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761314654.945606:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761314654.945619:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761314654.945667:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000eca34e1c x1846869307509568/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761314654.945695:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314654.945699:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761314654.945729:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ba5c8e77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307509504:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314654.945735:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e9376149 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307509440:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761314654.945858:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eca34e1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307509568:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761314654.951688:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ba5c8e77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307509504:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314654.952350:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e9376149 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307509440:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761314654.952449:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761314654.952454:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761314654.952458:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eca34e1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307509568:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761314654.953492:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039978a4e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307509376:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314655.202705:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314655.265670:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314656.226164:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314656.289287:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314657.248826:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314657.313195:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314658.273094:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314658.337226:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314659.296838:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314659.361833:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761314660.069113:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314660.069123:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761314660.069128:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1761314660.069142:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307509632:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761314660.069145:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2903 00000100:00080000:0.0:1761314660.069150:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761314660.069160:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761314660.069172:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761314660.069192:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ace4d025 x1846869307509824/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761314660.069239:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314660.069242:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761314660.069267:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307509696:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314660.069292:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307509824:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761314660.069442:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307509760:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761314660.075029:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761314660.075042:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761314660.075054:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307509824:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761314660.075320:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307509696:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314660.076518:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307509632:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314660.076537:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307509760:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314660.320833:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314660.385975:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314661.345243:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314661.409187:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314662.368797:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314662.432903:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314663.393055:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314663.456888:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314664.418176:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314664.481160:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761314665.184952:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314665.184960:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761314665.184963:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761314665.184974:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2908 00000100:00080000:1.0:1761314665.184977:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761314665.184987:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761314665.184997:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761314665.185011:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ed2f9bca x1846869307510080/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761314665.185031:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314665.185034:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761314665.185054:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307509952:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314665.185124:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307510080:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761314665.185146:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1761314665.185156:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307509888:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314665.185194:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307510016:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314665.188101:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307509888:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314665.190801:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307510016:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314665.190832:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307509952:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761314665.191849:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761314665.191856:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761314665.191864:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307510080:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761314665.442715:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314665.504878:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314666.466587:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314666.528865:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314667.488810:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314667.553772:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314668.512806:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314668.576883:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314669.538541:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314669.600823:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761314670.305205:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314670.305212:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761314670.305215:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761314670.305225:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2913 00000100:00080000:3.0:1761314670.305228:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761314670.305234:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761314670.305243:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761314670.305257:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006e93398e x1846869307510336/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761314670.305279:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314670.305282:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761314670.305307:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307510208:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314670.305968:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307510144:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761314670.306044:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307510336:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314670.306689:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307510272:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314670.312938:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307510208:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761314670.313533:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761314670.313538:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761314670.313543:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307510336:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314670.313914:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307510144:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314670.313927:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307510272:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314670.560892:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314670.624858:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314671.584780:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314671.649077:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314672.608758:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314672.672817:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314673.632749:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314673.696935:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314674.657015:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314674.721717:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761314675.425690:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314675.425697:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761314675.425701:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761314675.425711:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2918 00000100:00080000:2.0:1761314675.425714:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761314675.425720:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761314675.425729:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761314675.425760:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008a641e2e x1846869307510592/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761314675.425785:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314675.425789:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761314675.425815:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e3466f5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307510464:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761314675.425914:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008a641e2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307510592:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761314675.426391:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e9376149 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307510400:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314675.427777:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000015f165e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307510528:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761314675.432430:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761314675.432441:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761314675.432451:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008a641e2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307510592:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314675.432843:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e3466f5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307510464:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314675.433186:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e9376149 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307510400:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314675.433200:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000015f165e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307510528:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314675.685210:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314675.745946:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314676.704802:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314676.768877:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314677.728789:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314677.792841:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314678.752848:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314678.817309:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314679.776756:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314679.840864:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314680.545777:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e3466f5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307510656:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314680.547099:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761314680.547114:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e662bceb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307510720:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761314680.547385:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314680.547390:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761314680.547393:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761314680.547398:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2923 00000100:00080000:2.0:1761314680.547401:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761314680.547406:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761314680.547416:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761314680.547430:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ee08055d x1846869307510848/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761314680.547451:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314680.547455:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761314680.550693:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ee08055d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307510848:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761314680.551071:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761314680.551084:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000011b5c99f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307510784:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314680.551428:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e3466f5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307510656:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314680.554682:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e662bceb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307510720:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761314680.555073:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761314680.555078:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761314680.555082:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ee08055d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307510848:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761314680.556044:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000011b5c99f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307510784:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314680.800986:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314680.865122:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314681.824715:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314681.889515:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314682.848802:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314682.914162:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314683.872768:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314683.936877:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314684.896800:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314684.961618:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761314685.664877:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314685.664884:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761314685.664887:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761314685.664898:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2928 00000100:00080000:0.0:1761314685.664901:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761314685.664907:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761314685.664916:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761314685.664948:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000bcaee194 x1846869307511104/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761314685.664970:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314685.664973:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761314685.664996:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307510976:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314685.665086:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1761314685.665093:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307510912:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314685.665114:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307511040:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314685.668888:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307511104:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00080000:3.0:1761314685.672701:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761314685.672712:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761314685.672721:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307511104:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761314685.673353:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307510912:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314685.673394:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307511040:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314685.673405:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307510976:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314685.922633:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314685.984948:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314686.944834:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314687.011069:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314687.969115:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314688.033480:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314688.993465:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314689.057164:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314690.016819:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314690.080964:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761314690.788140:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314690.788149:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761314690.788153:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761314690.788161:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2933 00000100:00100000:1.0:1761314690.788161:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307511168:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761314690.788165:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761314690.788171:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761314690.788183:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761314690.788198:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000028de4b9d x1846869307511360/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761314690.788222:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314690.788226:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761314690.788256:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307511296:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314690.788267:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307511232:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314690.788291:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307511360:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761314690.796558:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307511232:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761314690.796634:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761314690.796659:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761314690.796664:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307511360:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761314690.797702:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307511168:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314690.797720:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307511296:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314691.041030:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314691.104909:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314692.066983:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314692.130390:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314693.088750:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314693.155278:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314694.112817:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314694.177193:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314695.136939:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314695.202538:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761314695.905037:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314695.905047:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761314695.905051:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761314695.905065:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2938 00000100:00100000:2.0:1761314695.905067:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307511424:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761314695.905070:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761314695.905080:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761314695.905093:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761314695.905135:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000602b09d0 x1846869307511616/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761314695.905161:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314695.905164:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761314695.905187:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307511488:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314695.905190:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307511552:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761314695.905805:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307511616:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761314695.914039:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307511488:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314695.915377:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307511424:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314695.915944:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307511552:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761314695.916058:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761314695.916064:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761314695.916069:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307511616:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314696.160933:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314696.225431:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314697.185270:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314697.249417:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314698.208864:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314698.275934:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314699.232827:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314699.298229:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314700.257054:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314700.323678:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761314701.024944:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314701.024956:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761314701.024960:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761314701.024976:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2943 00000100:00080000:0.0:1761314701.024980:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761314701.024987:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761314701.025000:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761314701.025018:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000005b900d12 x1846869307511872/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761314701.025043:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314701.025047:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761314701.025137:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307511872:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761314701.025943:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307511744:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314701.027634:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1761314701.027668:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307511680:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314701.031191:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307511808:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761314701.036539:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761314701.036553:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761314701.036565:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307511872:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761314701.049724:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307511680:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314701.049755:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307511808:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314701.049769:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307511744:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314701.280924:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314701.346781:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314702.304722:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314702.369411:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314703.329150:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314703.393179:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314704.352956:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314704.420341:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314705.376801:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314705.442232:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761314706.145030:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307511936:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314706.145782:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761314706.145791:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307512000:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761314706.146624:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314706.146631:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761314706.146665:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761314706.146673:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2949 00000100:00080000:0.0:1761314706.146676:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761314706.146683:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761314706.146697:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761314706.146717:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008e59d025 x1846869307512128/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761314706.146737:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314706.146740:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761314706.146771:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761314706.146784:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307512064:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314706.149240:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307512128:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761314706.151256:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307512000:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314706.152789:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307512064:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314706.152980:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307511936:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761314706.158842:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761314706.158858:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761314706.158870:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307512128:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761314706.403723:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314706.466126:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314707.424800:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314707.489549:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314708.448794:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314708.513433:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314709.472772:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314709.538053:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314710.497335:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314710.562506:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761314711.264927:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314711.264938:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761314711.264942:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761314711.264956:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2954 00000100:00080000:0.0:1761314711.264960:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761314711.264968:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761314711.264980:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761314711.265018:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000005a420586 x1846869307512384/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761314711.265046:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314711.265050:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761314711.266629:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307512192:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314711.266803:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307512320:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314711.266845:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761314711.266852:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307512256:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314711.267539:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307512384:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761314711.275673:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307512192:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761314711.282699:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761314711.282714:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761314711.282751:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307512384:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761314711.282894:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307512256:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314711.282909:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307512320:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314711.520815:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314711.585266:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314712.544790:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314712.611745:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314713.568744:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314713.635432:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314714.592815:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314714.657897:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314715.617681:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314715.681276:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314716.384916:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e1be9a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307512448:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761314716.386909:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314716.386914:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761314716.386918:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761314716.386926:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2959 00000100:00080000:2.0:1761314716.386928:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761314716.386933:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761314716.386955:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761314716.386968:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000011b5c99f x1846869307512640/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761314716.386981:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314716.386983:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761314716.387008:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ffb03e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307512512:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314716.387062:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761314716.387066:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b3aaddc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307512576:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761314716.387083:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000011b5c99f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307512640:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761314716.390980:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e1be9a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307512448:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314716.396133:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b3aaddc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307512576:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314716.396151:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ffb03e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307512512:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761314716.397132:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761314716.397139:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761314716.397146:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000011b5c99f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307512640:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314716.640918:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314716.706712:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314717.664771:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314717.729367:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314718.688789:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314718.752880:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314719.713470:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314719.777259:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314720.737062:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314720.800839:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761314721.504923:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314721.504934:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761314721.504938:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761314721.504956:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2964 00000100:00080000:2.0:1761314721.504960:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761314721.504968:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761314721.504981:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761314721.504998:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e662bceb x1846869307512896/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761314721.505112:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e662bceb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307512896:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761314721.506042:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ffb03e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307512704:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314721.506101:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ee08055d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307512832:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314721.506139:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761314721.506145:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b3aaddc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307512768:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761314721.507200:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314721.507210:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00080000:3.0:1761314721.518517:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761314721.518533:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761314721.518548:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e662bceb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307512896:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761314721.519064:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ffb03e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307512704:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314721.519087:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ee08055d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307512832:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314721.521684:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b3aaddc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307512768:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314721.760770:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314721.826986:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314722.785314:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314722.851720:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314723.808763:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314723.873684:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314724.833718:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314724.898923:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314725.856825:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314725.922733:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761314726.625962:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307512960:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314726.626515:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761314726.626522:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307513024:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761314726.626831:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314726.626835:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761314726.626838:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761314726.626844:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2969 00000100:00080000:0.0:1761314726.626847:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761314726.626851:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761314726.626861:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761314726.626875:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000fe4020c4 x1846869307513152/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761314726.626910:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314726.626913:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761314726.627108:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307513152:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761314726.628138:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761314726.628159:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307513088:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314726.632667:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307513024:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314726.632694:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307512960:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761314726.633827:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761314726.633836:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761314726.633860:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307513152:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761314726.634739:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307513088:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314726.880769:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314726.945852:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314727.905115:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314727.970358:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314728.928800:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314728.998434:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314729.952785:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314730.017255:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314730.976815:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314731.040982:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761314731.744958:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314731.744969:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761314731.744973:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761314731.744986:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2974 00000100:00080000:3.0:1761314731.744990:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761314731.744998:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761314731.745010:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761314731.745028:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006c8fcfe5 x1846869307513408/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761314731.745058:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314731.745062:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761314731.745099:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000626f98db pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307513280:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314731.745235:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1761314731.745242:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307513216:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314731.745270:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000038534f0b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307513344:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761314731.746168:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307513408:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761314731.749932:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000626f98db pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307513280:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314731.759015:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307513216:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761314731.765937:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:3.0:1761314731.765949:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000038534f0b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307513344:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761314731.765951:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761314731.765957:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307513408:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314732.001416:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314732.072796:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314733.029829:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314733.091200:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314734.050876:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314734.112952:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314735.072768:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314735.137476:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314736.096798:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314736.160871:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761314736.864918:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314736.864925:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761314736.864927:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761314736.864938:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2979 00000100:00080000:2.0:1761314736.864941:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761314736.864961:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761314736.864970:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761314736.864985:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006e3466f5 x1846869307513664/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761314736.865014:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314736.865017:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761314736.865043:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ee08055d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307513536:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314736.865134:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1761314736.865139:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b3aaddc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307513472:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314736.865161:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ffb03e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307513600:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761314736.865262:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e3466f5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307513664:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314736.871981:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ffb03e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307513600:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314736.872001:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ee08055d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307513536:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761314736.872190:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761314736.872195:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761314736.872200:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e3466f5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307513664:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314736.877188:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b3aaddc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307513472:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314737.120833:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314737.187432:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314738.146653:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314738.209781:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314739.168758:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314739.232915:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314740.193722:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314740.257185:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314741.216774:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314741.285482:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761314741.986354:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314741.986363:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761314741.986368:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761314741.986380:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2984 00000100:00080000:0.0:1761314741.986384:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761314741.986391:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761314741.986403:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761314741.986420:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000084f3d917 x1846869307513920/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761314741.986447:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314741.986467:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761314741.986499:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307513792:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314741.986606:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1761314741.986612:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307513728:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314741.986633:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307513856:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314741.987071:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307513920:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761314741.994120:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307513792:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314741.994165:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307513728:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314741.994175:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307513856:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761314741.994514:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761314741.994524:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761314741.994532:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307513920:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761314742.244412:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314742.305501:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314743.264922:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314743.330193:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314744.288798:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314744.354262:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314745.313196:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314745.376929:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314746.338340:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314746.404568:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314747.105186:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b3aaddc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307513984:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314747.106408:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ee08055d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307514048:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314747.107620:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ffb03e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307514112:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761314747.109124:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314747.109132:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761314747.109136:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761314747.109145:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2990 00000100:00080000:2.0:1761314747.109148:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761314747.109155:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761314747.109165:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761314747.109187:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000015f165e6 x1846869307514176/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1761314747.109235:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000015f165e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307514176:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00080000:2.0:1761314747.110235:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314747.110241:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761314747.111903:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ffb03e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307514112:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314747.111918:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b3aaddc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307513984:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314747.112095:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ee08055d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307514048:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761314747.113845:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761314747.113852:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761314747.113859:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000015f165e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307514176:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761314747.361154:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314747.424911:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314748.384800:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314748.448998:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314749.408879:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314749.472916:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314750.433508:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314750.497094:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314751.456794:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314751.522734:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761314752.224886:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314752.224895:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761314752.224900:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761314752.224912:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 2995 00000100:00080000:2.0:1761314752.224916:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761314752.224923:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761314752.224935:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761314752.224972:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000039978a4e x1846869307514432/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761314752.225001:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314752.225004:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761314752.225063:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039978a4e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307514432:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761314752.225121:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ee08055d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307514240:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314752.225219:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008a641e2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307514368:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314752.225272:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761314752.225278:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e9376149 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307514304:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314752.228440:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008a641e2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307514368:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:02020000:1.0:1761314752.230928:0:2262:0:(client.c:1361:ptlrpc_check_status()) 11-0: lustre-OST0000-osc-ffff951bc4d8c800: operation ost_connect to node 192.168.203.151@tcp failed: rc = -19 00000100:00100000:3.0:1761314752.230991:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e9376149 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307514304:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761314752.231004:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761314752.231008:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761314752.231009:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ee08055d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307514240:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761314752.231012:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039978a4e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307514432:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314752.480809:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314752.544876:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314753.504792:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314753.570788:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314754.528769:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314754.593476:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314755.554174:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314755.616869:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314756.576772:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314756.641486:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761314757.345111:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314757.345119:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761314757.345122:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761314757.345132:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3000 00000100:00080000:0.0:1761314757.345135:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761314757.345142:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761314757.345151:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761314757.345166:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ace4d025 x1846869307514688/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761314757.345192:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314757.345212:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761314757.345234:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307514560:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314757.345254:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307514688:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761314757.345333:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1761314757.345339:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307514496:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314757.345476:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307514624:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314757.348200:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307514496:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314757.348411:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307514624:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314757.348424:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307514560:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761314757.348460:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761314757.348466:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761314757.348470:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307514688:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761314757.601142:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314757.664968:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314758.626247:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314758.689301:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314759.649507:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314759.717242:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314760.674719:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314760.736877:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314761.696774:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314761.761255:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761314762.465822:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307514752:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314762.466431:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761314762.466440:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307514816:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761314762.466709:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314762.466714:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761314762.466717:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761314762.466723:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3005 00000100:00080000:0.0:1761314762.466726:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761314762.466732:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761314762.466740:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761314762.466767:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000fe4020c4 x1846869307514944/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761314762.466784:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314762.466787:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761314762.466838:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761314762.466848:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307514880:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314762.470248:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307514944:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761314762.470942:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307514816:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314762.470952:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307514752:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314762.472818:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307514880:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761314762.478208:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761314762.478222:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761314762.478236:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307514944:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761314762.720843:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314762.787744:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314763.746518:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314763.811776:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314764.768776:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314764.833538:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314765.792755:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314765.857273:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314766.816797:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314766.891519:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761314767.584938:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314767.584945:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761314767.584948:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761314767.584967:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3010 00000100:00080000:2.0:1761314767.584971:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761314767.584977:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761314767.584989:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761314767.585007:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000001426be00 x1846869307515200/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1761314767.585071:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eca34e1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307515008:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761314767.585119:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001426be00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307515200:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00080000:2.0:1761314767.585147:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314767.585151:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761314767.585191:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ba5c8e77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307515072:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314767.585208:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ae09c97d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307515136:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761314767.591920:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761314767.591935:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761314767.591947:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001426be00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307515200:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761314767.592005:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eca34e1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307515008:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314767.592019:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ae09c97d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307515136:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314767.592115:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ba5c8e77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307515072:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314767.840900:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314767.904944:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314768.864757:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314768.928868:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314769.888807:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314769.952900:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314770.913739:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314770.979513:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314771.936797:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314772.000918:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761314772.704904:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314772.704914:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761314772.704919:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761314772.704949:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3015 00000100:00080000:2.0:1761314772.704953:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00100000:3.0:1761314772.704962:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ba5c8e77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307515264:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761314772.704963:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761314772.704977:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761314772.704998:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b4eac89a x1846869307515456/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761314772.705027:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314772.705030:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761314772.705073:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b4eac89a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307515456:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761314772.705084:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002292ab57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307515392:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314772.705088:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307515328:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761314772.710027:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:3.0:1761314772.710037:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ba5c8e77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307515264:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761314772.710053:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761314772.710060:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b4eac89a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307515456:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761314772.710067:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002292ab57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307515392:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314772.710120:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307515328:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314772.960841:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314773.026099:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314773.984748:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314774.049411:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314775.012568:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314775.073061:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314776.032759:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314776.097720:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314777.056774:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314777.122851:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314777.827058:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000038534f0b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307515520:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314777.828672:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761314777.828685:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307515584:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761314777.828870:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314777.828875:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761314777.828878:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761314777.828883:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3020 00000100:00080000:3.0:1761314777.828894:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761314777.828900:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761314777.828908:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761314777.828922:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000004505177d x1846869307515712/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761314777.828956:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314777.828960:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761314777.829005:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761314777.829013:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000626f98db pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307515648:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314777.829085:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307515712:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761314777.831367:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000626f98db pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307515648:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314777.831455:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307515584:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761314777.831857:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761314777.831863:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761314777.831867:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307515712:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761314777.833512:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000038534f0b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307515520:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314778.080778:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314778.147075:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314779.104750:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314779.168891:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314780.129161:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314780.194115:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314781.154686:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314781.217396:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314782.176754:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314782.241163:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314782.945101:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000038534f0b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307515776:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314782.945316:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761314782.945324:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307515840:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761314782.945618:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314782.945623:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761314782.945626:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761314782.945632:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3025 00000100:00080000:3.0:1761314782.945634:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761314782.945657:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761314782.945666:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761314782.945681:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000cdcc674d x1846869307515968/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761314782.945702:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314782.945705:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761314782.945756:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761314782.945765:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004e00d332 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307515904:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314782.945882:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cdcc674d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307515968:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761314782.950475:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307515840:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314782.950493:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000038534f0b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307515776:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314782.951772:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004e00d332 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307515904:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761314782.954746:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761314782.954757:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761314782.954767:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cdcc674d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307515968:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761314783.202070:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314783.265917:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314784.224764:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314784.291669:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314785.248779:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314785.312888:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314786.272758:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314786.337293:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314787.299986:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314787.360917:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761314788.064907:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314788.064916:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761314788.064920:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761314788.064933:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3031 00000100:00080000:3.0:1761314788.064937:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761314788.064945:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761314788.064976:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761314788.065008:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cdcc674d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307516032:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314788.065025:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000039277a44 x1846869307516224/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761314788.065055:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314788.065062:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761314788.065158:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004e00d332 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307516096:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314788.065472:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000038534f0b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307516160:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314788.065675:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307516224:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761314788.069609:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cdcc674d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307516032:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314788.070851:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000038534f0b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307516160:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314788.070851:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004e00d332 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307516096:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761314788.073328:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761314788.073342:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761314788.073352:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307516224:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761314788.320860:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314788.386900:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314789.344835:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314789.409557:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314790.368840:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314790.432928:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314791.392803:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314791.458351:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314792.420954:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314792.480945:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761314793.187052:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314793.187063:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761314793.187086:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761314793.187101:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3036 00000100:00080000:0.0:1761314793.187105:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761314793.187112:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761314793.187125:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761314793.187147:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008e59d025 x1846869307516480/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761314793.187172:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314793.187176:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761314793.187279:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307516288:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761314793.187394:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307516416:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761314793.187427:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761314793.187432:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307516352:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314793.188817:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307516480:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761314793.191064:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307516352:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761314793.191086:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307516288:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761314793.191120:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307516416:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761314793.195044:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761314793.195062:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761314793.195075:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307516480:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314793.440838:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314793.505319:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314794.469170:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314794.529420:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314795.488895:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314795.552934:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314796.512805:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314796.578116:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314797.537762:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314797.604921:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761314798.305213:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314798.305223:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761314798.305227:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761314798.305239:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3041 00000100:00080000:3.0:1761314798.305243:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761314798.305250:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761314798.305261:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761314798.305279:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006c8fcfe5 x1846869307516736/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761314798.305336:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307516736:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314798.306569:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004e00d332 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307516544:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314798.306699:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000626f98db pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307516672:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314798.306737:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761314798.306790:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307516608:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761314798.311044:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314798.311055:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00080000:0.0:1761314798.314794:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761314798.314809:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761314798.314821:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307516736:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314798.320875:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004e00d332 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307516544:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314798.320904:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000626f98db pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307516672:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314798.320932:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307516608:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314798.561138:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314798.625840:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314799.591243:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314799.649130:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314800.608809:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314800.674127:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314801.633408:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314801.697403:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314802.656902:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314802.720951:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761314803.426841:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314803.426877:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761314803.426882:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761314803.426895:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3046 00000100:00080000:2.0:1761314803.426899:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761314803.426906:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761314803.426920:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761314803.426940:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000004592c0a5 x1846869307516992/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761314803.426968:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314803.426972:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761314803.427005:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b4eac89a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307516864:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314803.427008:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004592c0a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307516992:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761314803.427070:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307516800:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314803.427821:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b2298797 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307516928:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761314803.437473:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761314803.437489:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761314803.437501:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004592c0a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307516992:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761314803.437736:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b4eac89a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307516864:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314803.439357:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307516800:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314803.439393:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b2298797 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307516928:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314803.680816:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314803.746616:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314804.704848:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314804.770738:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314805.729532:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314805.793454:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314806.753957:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314806.818518:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314807.776761:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314807.843124:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761314808.544853:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314808.544861:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761314808.544865:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761314808.544876:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3051 00000100:00080000:1.0:1761314808.544879:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761314808.544885:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761314808.544894:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761314808.544908:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000009cec3faa x1846869307517248/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761314808.544977:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307517248:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00080000:1.0:1761314808.545525:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314808.545530:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761314808.545556:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307517120:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761314808.545596:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1761314808.545603:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307517056:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761314808.545628:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307517184:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761314808.548600:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307517184:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761314808.548616:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307517120:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761314808.550858:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761314808.550868:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761314808.550877:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307517248:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761314808.551143:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307517056:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314808.802315:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314808.864890:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314809.824775:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314809.893893:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314810.849970:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314810.914916:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314811.872783:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314811.938084:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314812.896780:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314812.961596:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761314813.664825:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314813.664832:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761314813.664848:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761314813.664857:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3056 00000100:00080000:1.0:1761314813.664861:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761314813.664866:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761314813.664877:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761314813.664892:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000540c3ce6 x1846869307517504/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761314813.664984:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314813.664987:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761314813.665712:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307517504:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761314813.667098:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307517312:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314813.667173:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307517440:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314813.667202:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761314813.667207:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307517376:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761314813.683107:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:0.0:1761314813.683108:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307517376:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761314813.683116:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761314813.683123:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307517504:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761314813.683130:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307517312:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314813.683143:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307517440:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314813.924246:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314813.986368:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314814.944852:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314815.008883:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314815.971508:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314816.033252:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314816.992797:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314817.059998:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314818.020739:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314818.080906:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761314818.784882:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307517568:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761314818.786197:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314818.786202:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761314818.786205:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761314818.786213:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3061 00000100:00080000:1.0:1761314818.786216:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761314818.786222:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761314818.786232:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761314818.786245:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a15ef099 x1846869307517760/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761314818.786257:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314818.786260:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761314818.786281:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307517696:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761314818.786329:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761314818.786335:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307517632:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314818.786473:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307517760:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761314818.793392:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307517568:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761314818.794028:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761314818.794033:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761314818.794039:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307517760:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761314818.794785:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307517696:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314818.794806:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307517632:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314819.045700:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314819.104860:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314820.065481:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314820.128871:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314821.091088:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314821.153337:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314822.112766:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314822.177355:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314823.138215:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314823.201816:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761314823.904884:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314823.904891:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761314823.904894:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761314823.904904:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3066 00000100:00080000:0.0:1761314823.904907:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761314823.904914:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761314823.904923:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761314823.904937:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000005b900d12 x1846869307518016/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761314823.904961:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314823.904964:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761314823.904987:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307517888:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761314823.905085:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1761314823.905091:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307517824:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761314823.905113:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307517952:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314823.905379:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307518016:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761314823.909908:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307517824:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761314823.909939:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307517952:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761314823.909953:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307517888:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761314823.913994:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761314823.914009:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761314823.914021:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307518016:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761314824.161211:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314824.227016:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314825.188832:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314825.250711:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314826.208912:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314826.273436:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314827.232783:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314827.298825:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314828.257434:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314828.322366:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761314829.025081:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314829.025090:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761314829.025094:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761314829.025107:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3071 00000100:00080000:3.0:1761314829.025110:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761314829.025116:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761314829.025128:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761314829.025163:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000d82b82f x1846869307518272/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761314829.025190:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314829.025193:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761314829.025221:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307518144:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314829.025248:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307518272:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761314829.025338:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1761314829.025351:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307518080:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314829.025382:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307518208:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761314829.029878:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761314829.029894:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761314829.029906:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307518272:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761314829.030045:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307518208:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314829.030060:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307518144:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314829.034697:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307518080:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314829.282851:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314829.345959:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314830.304835:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314830.369215:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314831.328861:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314831.394026:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314832.352768:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314832.420007:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314833.377340:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314833.440861:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761314834.145412:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314834.145420:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761314834.145438:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761314834.145449:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3077 00000100:00080000:2.0:1761314834.145452:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761314834.145457:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761314834.145467:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761314834.145483:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000007136630 x1846869307518528/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761314834.145520:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314834.145524:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761314834.145548:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aea3302e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307518400:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314834.145689:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1761314834.145695:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b4eac89a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307518336:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314834.145721:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f220504f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307518464:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314834.146050:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000007136630 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307518528:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00080000:0.0:1761314834.155912:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761314834.155930:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761314834.155941:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000007136630 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307518528:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761314834.156402:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b4eac89a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307518336:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314834.156431:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f220504f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307518464:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314834.156442:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aea3302e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307518400:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314834.403680:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314834.470700:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314835.424922:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314835.488834:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314836.450500:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314836.514409:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314837.473456:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314837.538423:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314838.496767:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314838.561128:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761314839.264857:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314839.264865:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761314839.264868:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761314839.264878:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3082 00000100:00080000:2.0:1761314839.264882:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761314839.264903:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761314839.264913:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761314839.264934:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000039eebc2 x1846869307518784/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761314839.265008:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000039eebc2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307518784:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761314839.265160:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aea3302e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307518592:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314839.265197:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b4eac89a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307518720:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314839.265229:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761314839.265236:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f220504f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307518656:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761314839.265540:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314839.265545:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761314839.269540:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aea3302e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307518592:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314839.269573:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b4eac89a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307518720:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314839.270329:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f220504f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307518656:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761314839.270585:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761314839.270591:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761314839.270597:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000039eebc2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307518784:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761314839.520810:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314839.589185:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314840.544807:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314840.608907:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314841.568961:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314841.635415:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314842.593241:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314842.657048:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314843.616789:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314843.680870:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761314844.384921:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314844.384928:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761314844.384932:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761314844.384943:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3087 00000100:00080000:3.0:1761314844.384946:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761314844.384952:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761314844.384961:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761314844.384975:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000004c6aebc5 x1846869307519040/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761314844.385001:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314844.385017:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761314844.385042:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307518912:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314844.385134:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1761314844.385141:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307518848:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314844.385167:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307518976:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314844.386941:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307519040:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00080000:0.0:1761314844.396391:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761314844.396401:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761314844.396409:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307519040:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761314844.396534:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307518848:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314844.396548:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307518976:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314844.396556:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307518912:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314844.640824:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314844.705293:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314845.666048:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314845.730718:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314846.688799:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314846.752860:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314847.712781:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314847.777432:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314848.736820:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314848.801128:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761314849.504928:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314849.504938:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761314849.504942:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761314849.504956:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3092 00000100:00080000:2.0:1761314849.504960:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761314849.504967:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761314849.504980:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761314849.504997:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000aea3302e x1846869307519296/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761314849.505022:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314849.505026:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761314849.505067:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f220504f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307519168:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314849.505077:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000039eebc2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307519104:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314849.505196:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b4eac89a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307519232:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761314849.505344:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aea3302e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307519296:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761314849.508106:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000039eebc2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307519104:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761314849.510558:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f220504f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307519168:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314849.512134:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b4eac89a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307519232:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761314849.512200:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761314849.512205:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761314849.512210:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aea3302e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307519296:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761314849.760787:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314849.824876:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314850.785165:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314850.848878:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314851.808779:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314851.874519:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314852.833499:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314852.897017:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314853.857113:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314853.920855:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761314854.624920:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314854.624931:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761314854.624935:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761314854.624948:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3097 00000100:00080000:2.0:1761314854.624952:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761314854.624959:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761314854.624972:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761314854.624988:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b8c05a37 x1846869307519552/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761314854.625081:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307519552:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00080000:2.0:1761314854.625509:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314854.625518:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761314854.625570:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f220504f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307519360:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314854.625811:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b2298797 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307519488:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314854.625856:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761314854.625885:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000007136630 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307519424:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314854.632756:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000007136630 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307519424:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314854.632774:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f220504f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307519360:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314854.632783:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b2298797 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307519488:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761314854.632890:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761314854.632895:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761314854.632900:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307519552:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761314854.880848:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314854.945064:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314855.904761:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314855.969180:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314856.928802:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314856.992912:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314857.954284:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314858.018614:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314858.976814:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314859.049404:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314859.744969:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004592c0a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307519616:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314859.747510:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002292ab57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307519680:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314859.750688:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004592c0a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307519616:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314859.751384:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ba5c8e77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307519744:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761314859.752494:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314859.752502:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761314859.752505:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761314859.752512:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3102 00000100:00080000:2.0:1761314859.752514:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761314859.752520:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761314859.752530:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761314859.752550:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ae09c97d x1846869307519808/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761314859.752597:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ae09c97d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307519808:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00080000:2.0:1761314859.753650:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314859.753671:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00080000:0.0:1761314859.757418:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:3.0:1761314859.757427:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002292ab57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307519680:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761314859.757482:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761314859.757488:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ae09c97d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307519808:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761314859.757562:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ba5c8e77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307519744:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761314860.000780:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314860.064962:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314861.024735:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314861.089156:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314862.050983:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314862.115135:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314863.073713:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314863.137351:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314864.096820:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314864.160864:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761314864.866665:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314864.866673:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761314864.866676:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761314864.866688:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3107 00000100:00080000:0.0:1761314864.866692:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761314864.866698:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761314864.866708:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761314864.866724:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e00b5e9d x1846869307520064/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761314864.866747:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314864.866750:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761314864.866863:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307519872:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761314864.866953:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307520000:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761314864.866997:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761314864.867003:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307519936:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314864.867416:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307520064:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761314864.871838:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307519936:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761314864.871859:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307520000:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761314864.873147:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761314864.873156:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761314864.873163:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307520064:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761314864.874218:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307519872:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314865.120765:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314865.185531:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314866.146173:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314866.211345:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314867.169443:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314867.234099:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314868.194036:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314868.257408:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314869.220411:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314869.280979:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761314869.984909:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314869.984919:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761314869.984923:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761314869.984936:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3112 00000100:00080000:3.0:1761314869.984940:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761314869.984947:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761314869.984964:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761314869.984984:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a183f7ef x1846869307520320/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761314869.985010:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314869.985013:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761314869.985048:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307520320:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314869.985117:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307520128:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314869.985172:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cc6dd8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307520256:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314869.985204:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761314869.985209:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f133bb2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307520192:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761314869.989762:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761314869.989775:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761314869.989785:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307520320:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314869.990008:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f133bb2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307520192:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314869.990028:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307520128:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314869.990040:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc6dd8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307520256:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314870.240791:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314870.304873:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314871.267315:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314871.329014:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314872.288813:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314872.352936:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314873.312771:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314873.378455:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314874.336789:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314874.400884:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761314875.104854:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f597cda6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307520384:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314875.106411:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307520448:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314875.108572:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000101f852a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307520512:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761314875.109564:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314875.109569:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761314875.109573:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761314875.109579:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3118 00000100:00080000:3.0:1761314875.109582:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761314875.109587:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761314875.109597:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761314875.109616:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000bc14d8e x1846869307520576/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761314875.110186:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307520576:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00080000:3.0:1761314875.112522:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314875.112531:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761314875.115836:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f597cda6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307520384:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314875.115959:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307520448:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314875.117674:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000101f852a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307520512:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761314875.117913:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761314875.117918:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761314875.117923:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307520576:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761314875.363340:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314875.426124:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314876.388315:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314876.451281:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314877.409259:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314877.472855:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314878.432748:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314878.496852:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314879.456795:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314879.521527:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761314880.224937:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307520640:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761314880.227019:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307520704:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761314880.230241:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307520768:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761314880.230926:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314880.230932:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761314880.230936:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761314880.230943:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3123 00000100:00080000:1.0:1761314880.230946:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761314880.230952:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761314880.230981:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761314880.231001:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a1719d14 x1846869307520832/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761314880.231065:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307520832:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00080000:1.0:1761314880.232351:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314880.232357:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00080000:2.0:1761314880.236575:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761314880.236587:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761314880.236597:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307520832:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761314880.237850:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307520640:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761314880.237941:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307520768:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761314880.238030:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307520704:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761314880.482760:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314880.546429:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314881.504813:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314881.569251:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314882.530117:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314882.592937:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314883.552854:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314883.617980:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314884.577170:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314884.642956:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761314885.346762:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314885.346769:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761314885.346772:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761314885.346782:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3128 00000100:00080000:0.0:1761314885.346785:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761314885.346791:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761314885.346800:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761314885.346814:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e00b5e9d x1846869307521088/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761314885.346838:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314885.346842:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761314885.346915:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307520896:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761314885.346994:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307521024:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761314885.347021:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761314885.347026:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307520960:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761314885.347867:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307521088:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761314885.354708:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307521024:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761314885.354737:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307520960:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761314885.359206:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307520896:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761314885.359309:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761314885.359314:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761314885.359318:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307521088:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761314885.600804:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314885.664909:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314886.626521:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314886.688996:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314887.648799:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314887.716470:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314888.674685:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314888.737201:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314889.696804:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314889.768683:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761314890.465710:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314890.465718:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761314890.465721:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761314890.465731:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3133 00000100:00080000:1.0:1761314890.465734:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761314890.465751:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761314890.465760:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761314890.465775:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000003871442b x1846869307521344/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761314890.465796:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314890.465799:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761314890.465863:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307521152:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761314890.465937:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307521344:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761314890.465950:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307521280:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761314890.465996:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761314890.466002:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307521216:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761314890.477522:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307521152:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761314890.479843:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761314890.479850:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761314890.479857:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307521344:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761314890.480001:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307521216:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761314890.480028:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307521280:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761314890.721111:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314890.785331:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314891.745347:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314891.809336:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314892.769688:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314892.835036:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314893.792742:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314893.857285:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314894.816926:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314894.880900:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761314895.584955:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314895.584966:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761314895.584970:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761314895.584983:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3138 00000100:00080000:0.0:1761314895.584988:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761314895.584994:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761314895.585007:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761314895.585026:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000005b900d12 x1846869307521600/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761314895.585059:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314895.585063:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761314895.585096:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307521472:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761314895.585209:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1761314895.585216:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307521408:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761314895.585249:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307521536:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761314895.585543:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307521600:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761314895.593932:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307521408:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761314895.593957:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307521536:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761314895.593967:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307521472:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761314895.594186:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761314895.594191:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761314895.594197:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307521600:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761314895.846235:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314895.905884:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314896.864803:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314896.929576:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314897.892088:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314897.952885:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314898.912797:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314898.979155:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314899.936811:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314900.000845:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761314900.704953:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314900.704963:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761314900.704967:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761314900.704979:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3143 00000100:00100000:0.0:1761314900.704980:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307521664:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761314900.704984:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761314900.704990:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761314900.705001:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761314900.705221:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307521792:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761314900.705254:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ed2f9bca x1846869307521856/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761314900.705259:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761314900.705264:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307521728:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761314900.705329:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307521856:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00080000:1.0:1761314900.705522:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314900.705534:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761314900.711074:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307521728:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761314900.711094:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307521664:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761314900.711105:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307521792:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761314900.714703:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761314900.714747:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761314900.714760:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307521856:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761314900.960740:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314901.025510:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314901.984814:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314902.048853:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314903.008799:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314903.073054:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314904.037151:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314904.099050:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314905.058281:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314905.121038:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761314905.824953:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307521920:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761314905.826671:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307521984:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761314905.827573:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761314905.827588:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307522048:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761314905.828368:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314905.828377:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761314905.828381:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761314905.828390:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3148 00000100:00080000:0.0:1761314905.828393:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761314905.828400:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761314905.828413:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761314905.828446:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007dcceb4a x1846869307522112/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761314905.828548:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307522112:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00080000:0.0:1761314905.828623:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314905.828628:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761314905.831495:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307522048:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761314905.831524:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307521984:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761314905.835753:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307521920:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:02020000:2.0:1761314905.844953:0:2262:0:(client.c:1361:ptlrpc_check_status()) 11-0: lustre-OST0000-osc-ffff951bc4d8c800: operation ost_connect to node 192.168.203.151@tcp failed: rc = -19 00000100:00080000:2.0:1761314905.844969:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761314905.844975:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761314905.844990:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307522112:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761314906.082251:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314906.144938:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314907.105508:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314907.170185:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314908.134139:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314908.193615:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314909.155296:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314909.217595:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314910.176777:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314910.241721:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761314910.944859:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314910.944869:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761314910.944872:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761314910.944884:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3153 00000100:00080000:1.0:1761314910.944888:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761314910.944895:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761314910.944907:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761314910.944923:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000003871442b x1846869307522368/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761314910.944977:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314910.945003:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761314910.945049:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307522176:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761314910.945103:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307522240:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761314910.945147:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307522304:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761314910.945208:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307522368:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761314910.952777:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307522304:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761314910.953267:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761314910.953275:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761314910.953281:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307522368:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761314910.954107:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307522240:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761314910.955796:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307522176:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761314911.202206:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314911.265009:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314912.224849:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314912.288979:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314913.248807:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314913.313502:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314914.272784:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314914.336868:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314915.296762:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314915.361697:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761314916.066034:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314916.066044:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761314916.066048:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761314916.066060:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3159 00000100:00080000:3.0:1761314916.066064:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761314916.066072:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761314916.066090:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761314916.066109:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e58cc722 x1846869307522624/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761314916.066164:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314916.066168:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761314916.066218:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307522432:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761314916.066884:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e58cc722 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307522624:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314916.067198:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d32bed84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307522560:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314916.067236:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761314916.067242:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000009911ad3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307522496:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761314916.071454:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761314916.071469:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761314916.071481:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e58cc722 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307522624:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314916.071572:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307522432:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314916.074207:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000009911ad3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307522496:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314916.074227:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d32bed84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307522560:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314916.321079:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314916.385562:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314917.344959:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314917.409274:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314918.369480:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314918.436040:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314919.392784:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314919.456893:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314920.416810:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314920.482214:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761314921.184947:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314921.184956:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761314921.184960:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761314921.184973:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3164 00000100:00080000:1.0:1761314921.184977:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761314921.184985:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761314921.184998:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761314921.185017:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000163e3801 x1846869307522880/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761314921.185038:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314921.185041:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761314921.185076:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307522752:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761314921.185184:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1761314921.185189:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307522688:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761314921.185230:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307522816:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761314921.185517:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307522880:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761314921.187541:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307522688:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761314921.187574:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307522816:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761314921.187586:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307522752:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761314921.189170:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761314921.189183:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761314921.189194:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307522880:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761314921.440778:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314921.511831:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314922.466186:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314922.529728:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314923.488810:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314923.552922:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314924.514088:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314924.581630:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314925.536771:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314925.600904:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761314926.309053:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314926.309061:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761314926.309064:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761314926.309073:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3169 00000100:00080000:1.0:1761314926.309077:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761314926.309083:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761314926.309092:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761314926.309109:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a1719d14 x1846869307523136/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761314926.309130:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314926.309133:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761314926.309700:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307523136:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761314926.310731:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307522944:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761314926.310855:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307523072:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761314926.310891:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761314926.310897:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307523008:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761314926.315212:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307523008:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761314926.315233:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307522944:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761314926.315245:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307523072:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761314926.316971:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761314926.317007:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761314926.317019:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307523136:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761314926.560799:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314926.628046:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314927.584795:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314927.649358:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314928.609446:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314928.673277:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314929.632931:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314929.697398:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314930.658425:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314930.724285:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761314931.425393:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314931.425401:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761314931.425404:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761314931.425415:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3174 00000100:00080000:3.0:1761314931.425418:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761314931.425423:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761314931.425434:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761314931.425449:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f364ec74 x1846869307523392/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761314931.425475:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314931.425479:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761314931.425568:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f364ec74 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307523392:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314931.426265:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e58cc722 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307523200:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314931.426313:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007673f690 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307523328:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314931.426338:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761314931.426344:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000041c8b6cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307523264:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314931.442398:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000041c8b6cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307523264:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314931.442447:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e58cc722 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307523200:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314931.442460:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007673f690 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307523328:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761314931.442583:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761314931.442590:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761314931.442595:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f364ec74 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307523392:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761314931.680789:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314931.746332:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314932.704782:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314932.768943:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314933.728771:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314933.794460:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314934.752765:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314934.819792:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314935.780946:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314935.840973:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761314936.544924:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314936.544933:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761314936.544937:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761314936.544951:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3179 00000100:00080000:2.0:1761314936.544956:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761314936.544963:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761314936.544976:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761314936.544995:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ee08055d x1846869307523648/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761314936.545024:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314936.545027:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761314936.545060:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eca34e1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307523520:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761314936.545179:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1761314936.545187:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ba5c8e77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307523456:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761314936.545214:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001426be00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307523584:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761314936.545382:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ee08055d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307523648:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761314936.552125:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ba5c8e77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307523456:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761314936.554328:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761314936.554343:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761314936.554353:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ee08055d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307523648:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761314936.554794:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001426be00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307523584:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761314936.554813:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eca34e1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307523520:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761314936.800826:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314936.868123:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314937.824912:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314937.888935:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314938.848947:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314938.914317:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314939.872844:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314939.938001:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314940.896771:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314940.961762:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761314941.664908:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314941.664919:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761314941.664923:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761314941.664936:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3184 00000100:00080000:2.0:1761314941.664941:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761314941.664948:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761314941.664961:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761314941.664977:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e9376149 x1846869307523904/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761314941.665005:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314941.665009:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761314941.665082:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e9376149 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307523904:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761314941.665125:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eca34e1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307523712:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761314941.665192:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ba5c8e77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307523840:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761314941.665225:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761314941.665231:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001426be00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307523776:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761314941.676254:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761314941.676269:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761314941.676281:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e9376149 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307523904:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761314941.682413:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eca34e1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307523712:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761314941.687203:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001426be00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307523776:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761314941.687230:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ba5c8e77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307523840:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761314941.921051:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314941.985081:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314942.944795:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314943.010942:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314943.971459:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314944.034264:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314944.992868:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314945.056863:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314946.016963:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314946.081165:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761314946.787095:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314946.787105:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761314946.787109:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761314946.787123:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3189 00000100:00080000:0.0:1761314946.787127:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761314946.787153:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761314946.787166:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761314946.787186:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000fe4020c4 x1846869307524160/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761314946.787212:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314946.787216:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761314946.787252:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307524032:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761314946.787408:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307523968:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314946.787541:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307524160:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761314946.787596:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307524096:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761314946.791343:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307524032:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761314946.791466:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307523968:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761314946.791480:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307524096:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761314946.791558:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761314946.791564:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761314946.791568:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307524160:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761314947.040822:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314947.104872:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314948.065488:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314948.128995:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314949.089096:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314949.154498:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314950.114837:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314950.179917:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314951.136805:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314951.201253:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761314951.904924:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314951.904944:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761314951.904948:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761314951.904957:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3194 00000100:00080000:1.0:1761314951.904961:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761314951.904967:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761314951.904976:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761314951.904992:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a15ef099 x1846869307524416/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761314951.904996:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307524224:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761314951.905020:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314951.905024:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761314951.905045:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307524288:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761314951.905081:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307524352:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761314951.905148:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307524416:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761314951.913715:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307524288:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761314951.913787:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307524352:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761314951.913891:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761314951.913897:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761314951.913902:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307524416:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761314951.919896:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307524224:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761314952.161565:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314952.224866:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314953.185062:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761314953.248853:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314954.208804:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314954.275618:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314955.233580:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314955.297241:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314956.256930:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314956.324752:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761314957.024973:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314957.024983:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761314957.025005:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761314957.025019:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3199 00000100:00080000:3.0:1761314957.025023:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761314957.025040:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761314957.025053:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761314957.025073:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000041c8b6cb x1846869307524672/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761314957.025096:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314957.025100:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761314957.025134:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007673f690 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307524544:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761314957.025190:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000041c8b6cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307524672:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761314957.025259:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1761314957.025270:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f364ec74 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307524480:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314957.025310:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e58cc722 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307524608:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761314957.027778:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:3.0:1761314957.027782:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f364ec74 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307524480:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761314957.027789:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761314957.027797:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e58cc722 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307524608:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314957.027805:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007673f690 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307524544:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761314957.027810:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000041c8b6cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307524672:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761314957.285053:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314957.345813:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314958.304768:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314958.369089:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314959.329195:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314959.392933:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314960.352772:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314960.417461:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314961.377544:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314961.440842:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761314962.144954:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307524736:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761314962.145163:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761314962.145170:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307524800:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761314962.145379:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314962.145384:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761314962.145387:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761314962.145392:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3205 00000100:00080000:0.0:1761314962.145394:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761314962.145399:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761314962.145409:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761314962.145440:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a24c5c6a x1846869307524928/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761314962.145456:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761314962.145459:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761314962.145507:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761314962.145515:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307524864:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314962.145851:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307524928:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761314962.155976:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307524736:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761314962.156015:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307524800:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761314962.156053:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307524864:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761314962.156387:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761314962.156392:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761314962.156397:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307524928:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761314962.402959:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314962.465076:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314963.424754:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314963.489582:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314964.448755:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314964.513216:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314965.473621:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314965.536821:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314966.496766:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314966.566695:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761314967.265111:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314967.265118:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761314967.265120:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761314967.265129:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3210 00000100:00080000:3.0:1761314967.265132:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761314967.265138:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761314967.265147:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761314967.265160:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d32bed84 x1846869307525184/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761314967.265183:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314967.265186:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761314967.265443:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d32bed84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307525184:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314967.265623:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e58cc722 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307525056:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314967.265701:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1761314967.265708:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007673f690 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307524992:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314967.265731:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f364ec74 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307525120:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761314967.273114:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761314967.273124:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761314967.273133:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d32bed84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307525184:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761314967.273830:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007673f690 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307524992:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314967.273880:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f364ec74 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307525120:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314967.273894:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e58cc722 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307525056:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314967.520811:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314967.584873:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314968.544791:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314968.608885:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314969.571688:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314969.634850:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314970.592870:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314970.657310:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314971.617020:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314971.680917:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761314972.388934:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314972.388941:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761314972.388945:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761314972.388957:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3215 00000100:00080000:2.0:1761314972.388961:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00100000:3.0:1761314972.388969:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039978a4e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307525248:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761314972.388970:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761314972.388983:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761314972.389021:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008b3aaddc x1846869307525440/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761314972.389049:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314972.389052:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761314972.389076:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008a641e2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307525312:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761314972.389086:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000015f165e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307525376:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761314972.389097:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b3aaddc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307525440:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761314972.395914:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000015f165e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307525376:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761314972.396042:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008a641e2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307525312:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761314972.398685:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039978a4e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307525248:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761314972.398780:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761314972.398785:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761314972.398790:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b3aaddc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307525440:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761314972.641117:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314972.706046:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314973.664827:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314973.728971:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314974.688771:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314974.753219:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314975.712866:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314975.782345:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314976.736772:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314976.800865:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761314977.505143:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314977.505150:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761314977.505153:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761314977.505164:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3220 00000100:00080000:1.0:1761314977.505167:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761314977.505173:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000100:00100000:0.0:1761314977.505178:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307525504:192.168.203.151@tcp:400:kworker/u8:3.0 00000008:00100000:1.0:1761314977.505184:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761314977.505202:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ed2f9bca x1846869307525696/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761314977.505233:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314977.505237:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761314977.505258:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307525568:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761314977.505277:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307525632:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761314977.505717:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307525696:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00080000:3.0:1761314977.514569:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761314977.514584:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761314977.514595:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307525696:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761314977.514712:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307525504:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761314977.515078:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307525568:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761314977.515112:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307525632:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761314977.760807:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314977.825534:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314978.784800:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314978.849975:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314979.809235:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314979.872872:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314980.832999:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314980.896795:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314981.856800:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314981.921610:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761314982.624872:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314982.624879:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761314982.624883:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761314982.624893:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3225 00000100:00100000:2.0:1761314982.624896:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e58cc722 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307525760:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761314982.624898:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761314982.624907:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761314982.624917:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761314982.624941:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000009911ad3 x1846869307525952/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761314982.624968:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314982.624972:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761314982.624991:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007673f690 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307525888:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314982.624996:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f364ec74 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307525824:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761314982.625126:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000009911ad3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307525952:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314982.634069:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e58cc722 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307525760:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314982.634096:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007673f690 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307525888:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314982.634851:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f364ec74 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307525824:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761314982.636976:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761314982.636990:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761314982.637001:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000009911ad3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307525952:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761314982.880839:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314982.945242:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314983.904824:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314983.969090:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314984.928801:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314984.992876:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314985.953011:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314986.016985:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314986.976834:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314987.040862:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761314987.745310:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314987.745318:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761314987.745320:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761314987.745330:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3230 00000100:00080000:2.0:1761314987.745333:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761314987.745339:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761314987.745348:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761314987.745362:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e662bceb x1846869307526208/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761314987.745385:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761314987.745440:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761314987.745468:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ffb03e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307526080:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761314987.745478:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e662bceb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307526208:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761314987.745590:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008a641e2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307526016:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761314987.745726:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e3466f5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307526144:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761314987.749435:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008a641e2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307526016:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761314987.749467:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ffb03e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307526080:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761314987.749468:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e3466f5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307526144:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761314987.751443:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761314987.751452:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761314987.751459:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e662bceb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307526208:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761314988.000948:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314988.065634:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314989.024842:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314989.088915:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314990.049095:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314990.113271:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314991.072764:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314991.136854:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314992.097073:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314992.160888:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761314992.865294:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314992.865303:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761314992.865307:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761314992.865335:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3235 00000100:00080000:1.0:1761314992.865340:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00100000:0.0:1761314992.865511:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307526272:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761314992.865613:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307526400:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761314992.865707:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761314992.865713:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307526336:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761314992.866051:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761314992.866064:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761314992.866085:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ed2f9bca x1846869307526464/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761314992.866112:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761314992.866116:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761314992.867072:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307526464:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761314992.868773:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307526336:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761314992.868795:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307526272:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761314992.868808:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307526400:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761314992.873867:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761314992.873899:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761314992.873910:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307526464:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761314993.120803:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314993.185688:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314994.144799:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314994.208892:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314995.168793:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314995.232871:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314996.198470:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314996.258355:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314997.216768:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314997.280900:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761314997.986904:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314997.986913:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761314997.986918:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761314997.986930:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3240 00000100:00080000:3.0:1761314997.986934:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761314997.986942:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761314997.986954:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761314997.986974:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000101f852a x1846869307526720/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761314997.987002:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761314997.987006:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761314997.987042:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307526592:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761314997.987058:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000101f852a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307526720:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314997.987422:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f364ec74 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307526528:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314997.989611:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307526656:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314997.991978:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307526592:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761314997.992275:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761314997.992280:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761314997.992285:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000101f852a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307526720:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761314997.993244:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307526656:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761314997.996818:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f364ec74 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307526528:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761314998.240825:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761314998.304919:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761314999.265231:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761314999.328971:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315000.289002:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315000.353337:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315001.312808:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315001.378498:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315002.336787:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315002.401466:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761315003.105823:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315003.105832:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761315003.105837:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761315003.105851:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3246 00000100:00080000:1.0:1761315003.105856:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761315003.105862:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761315003.105873:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761315003.105890:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a1719d14 x1846869307526976/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761315003.105929:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315003.105932:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761315003.105967:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307526848:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315003.106088:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1761315003.106094:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307526784:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315003.106114:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307526912:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761315003.106895:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307526976:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761315003.116918:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307526848:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315003.116962:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307526784:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315003.116975:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307526912:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761315003.117033:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761315003.117039:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761315003.117045:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307526976:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761315003.360820:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315003.425007:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315004.385736:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315004.449577:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315005.408932:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315005.473628:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315006.432810:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315006.497814:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315007.456746:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315007.520873:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761315008.224834:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315008.224842:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761315008.224845:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761315008.224856:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3251 00000100:00080000:3.0:1761315008.224859:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761315008.224866:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761315008.224876:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761315008.224893:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006e93398e x1846869307527232/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761315008.224915:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f597cda6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307527104:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315008.224983:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307527232:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315008.225071:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1761315008.225088:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307527040:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315008.225133:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cc6dd8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307527168:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761315008.225620:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315008.225627:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00080000:0.0:1761315008.232392:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761315008.232402:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761315008.232412:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307527232:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315008.235810:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f597cda6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307527104:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315008.235851:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc6dd8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307527168:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315008.237465:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307527040:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315008.481250:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315008.551124:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315009.505742:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315009.570191:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315010.529975:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315010.594053:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315011.552873:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315011.618556:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315012.579210:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315012.643837:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761315013.345019:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315013.345029:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00100000:0.0:1761315013.345031:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307527296:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761315013.345032:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761315013.345039:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3256 00000100:00080000:1.0:1761315013.345043:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761315013.345050:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761315013.345062:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761315013.345082:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a1719d14 x1846869307527488/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761315013.345106:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315013.345109:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761315013.345149:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307527424:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315013.345149:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307527360:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315013.345758:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307527488:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761315013.348436:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307527424:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315013.351612:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307527296:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315013.351791:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307527360:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761315013.352139:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761315013.352145:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761315013.352150:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307527488:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315013.600975:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315013.667100:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315014.624823:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315014.692590:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315015.648850:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315015.713002:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315016.672856:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315016.740009:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315017.696776:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315017.768310:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761315018.467228:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315018.467239:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761315018.467243:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761315018.467256:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3261 00000100:00080000:3.0:1761315018.467261:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761315018.467268:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761315018.467281:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761315018.467302:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000bfffbafe x1846869307527744/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761315018.467313:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f133bb2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307527552:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761315018.467338:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315018.467343:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761315018.467394:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307527616:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761315018.467425:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307527680:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315018.468316:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307527744:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761315018.480285:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307527616:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761315018.480333:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f133bb2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307527552:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761315018.480343:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307527680:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761315018.480446:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761315018.480456:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761315018.480465:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307527744:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761315018.720766:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315018.784856:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315019.744794:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315019.809152:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315020.768810:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315020.833022:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315021.792786:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315021.861368:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315022.816801:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315022.882414:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761315023.584893:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315023.584903:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761315023.584906:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761315023.584921:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3266 00000100:00080000:3.0:1761315023.584924:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761315023.584933:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761315023.584947:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761315023.584966:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000d82b82f x1846869307528000/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761315023.584996:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315023.585000:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761315023.585102:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307528000:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315023.586101:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307527808:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315023.586158:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307527936:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315023.586184:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761315023.586189:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307527872:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761315023.590245:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761315023.590262:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761315023.590275:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307528000:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315023.590987:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307527872:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315023.591004:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307527808:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315023.591014:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307527936:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315023.840769:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315023.905947:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315024.864785:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315024.929049:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315025.888757:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315025.953436:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315026.913150:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315026.984232:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315027.936726:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315028.000885:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761315028.706293:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315028.706300:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761315028.706304:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761315028.706314:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3271 00000100:00080000:2.0:1761315028.706317:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761315028.706323:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761315028.706332:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761315028.706347:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e662bceb x1846869307528256/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761315028.706372:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315028.706376:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761315028.706400:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000011b5c99f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307528128:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315028.706410:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e662bceb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307528256:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761315028.706739:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ffb03e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307528064:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761315028.707101:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e1be9a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307528192:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761315028.719127:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000011b5c99f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307528128:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761315028.724213:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761315028.724223:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761315028.724232:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e662bceb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307528256:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761315028.724527:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ffb03e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307528064:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761315028.724542:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e1be9a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307528192:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761315028.960770:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315029.029087:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315029.984773:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315030.048925:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315031.008780:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315031.072868:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315032.032795:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315032.096880:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315033.056764:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315033.121419:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761315033.825339:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315033.825351:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761315033.825355:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761315033.825369:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3276 00000100:00080000:3.0:1761315033.825374:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761315033.825382:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761315033.825397:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761315033.825417:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000004e00d332 x1846869307528512/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761315033.825439:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315033.825443:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761315033.825475:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307528384:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761315033.825594:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1761315033.825600:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307528320:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761315033.825634:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000626f98db pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307528448:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315033.825970:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004e00d332 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307528512:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761315033.829411:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307528320:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761315033.829449:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000626f98db pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307528448:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761315033.829460:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307528384:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761315033.833852:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761315033.833863:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761315033.833873:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004e00d332 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307528512:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761315034.082121:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315034.157301:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315035.104774:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315035.168863:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315036.129235:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315036.192877:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315037.152758:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315037.216873:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315038.177556:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315038.241897:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761315038.944891:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315038.944901:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761315038.944905:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761315038.944918:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3281 00000100:00080000:3.0:1761315038.944922:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 6s 00000100:00080000:3.0:1761315038.944929:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761315038.944942:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761315038.944961:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006c8fcfe5 x1846869307528768/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761315038.944986:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315038.944990:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761315038.945023:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000626f98db pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307528640:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761315038.945178:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1761315038.945185:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307528576:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761315038.945208:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307528704:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315038.945729:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307528768:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761315038.948818:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000626f98db pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307528640:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761315038.950928:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307528576:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761315038.952219:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761315038.952228:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761315038.952236:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307528768:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761315038.952363:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307528704:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761315039.205374:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315039.265178:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315040.228893:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315040.290394:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315041.248800:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315041.312913:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315042.272806:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315042.336954:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315043.296811:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315043.361390:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315044.066442:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307528832:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761315044.070970:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315044.070981:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761315044.070985:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761315044.071011:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3287 00000100:00080000:3.0:1761315044.071016:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 11s 00000100:00080000:3.0:1761315044.071023:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761315044.071038:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761315044.071058:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000039277a44 x1846869307529024/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761315044.071074:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315044.071078:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761315044.071185:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307529024:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315044.071569:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307528896:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315044.071628:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761315044.071634:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000626f98db pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307528960:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761315044.083325:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761315044.083339:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761315044.083352:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307529024:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315044.083352:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307528832:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315044.084026:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000626f98db pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307528960:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315044.084040:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307528896:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315044.321222:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315044.386159:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315045.344788:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315045.412878:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315046.369501:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315046.434141:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315047.394588:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315047.457973:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315048.417203:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315048.481013:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315049.185398:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000038534f0b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307529088:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761315049.185424:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315049.185433:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761315049.185438:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761315049.185451:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3292 00000100:00080000:3.0:1761315049.185476:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 16s 00000100:00080000:3.0:1761315049.185484:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761315049.185497:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761315049.185519:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000626f98db x1846869307529280/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761315049.185594:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000626f98db pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307529280:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315049.185881:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307529216:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315049.186492:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761315049.186501:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cdcc674d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307529152:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761315049.186716:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315049.186724:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00080000:0.0:1761315049.191175:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761315049.191185:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761315049.191212:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000626f98db pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307529280:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315049.191450:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307529216:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315049.191462:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000038534f0b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307529088:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315049.201269:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cdcc674d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307529152:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315049.445834:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315049.505578:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315050.468136:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315050.531129:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315051.489432:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315051.552945:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315052.513352:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315052.578203:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315053.536811:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315053.602224:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761315054.304919:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315054.304959:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761315054.304964:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761315054.304978:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3297 00000100:00080000:3.0:1761315054.304984:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761315054.304991:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761315054.305003:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761315054.305023:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000004e00d332 x1846869307529536/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761315054.305050:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315054.305054:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761315054.305358:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307529344:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315054.305480:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307529472:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315054.305520:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761315054.305527:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307529408:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315054.305923:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004e00d332 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307529536:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00080000:0.0:1761315054.315488:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761315054.315508:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761315054.315520:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004e00d332 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307529536:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315054.315537:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307529408:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315054.315548:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307529344:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315054.315567:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307529472:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315054.560968:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315054.625073:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315055.585317:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315055.649303:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315056.611991:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315056.676635:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315057.633087:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315057.696941:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315058.656862:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315058.727105:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761315059.425727:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315059.425736:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761315059.425758:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761315059.425771:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3302 00000100:00080000:2.0:1761315059.425775:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761315059.425782:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761315059.425794:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761315059.425799:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000011b5c99f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307529600:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761315059.425816:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008b3aaddc x1846869307529792/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761315059.425844:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315059.425847:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761315059.425872:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e3466f5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307529664:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761315059.425996:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b3aaddc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307529792:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761315059.426488:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008a641e2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307529728:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:02020000:0.0:1761315059.435592:0:2262:0:(client.c:1361:ptlrpc_check_status()) 11-0: lustre-OST0000-osc-ffff951bc4d8c800: operation ost_connect to node 192.168.203.151@tcp failed: rc = -19 00000100:00100000:2.0:1761315059.435856:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e3466f5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307529664:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761315059.436837:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000011b5c99f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307529600:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761315059.436859:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008a641e2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307529728:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761315059.448367:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761315059.448377:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761315059.448388:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b3aaddc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307529792:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761315059.681491:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315059.745048:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315060.704950:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315060.774188:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315061.730237:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315061.792915:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315062.752794:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315062.818424:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315063.777020:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315063.843267:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315064.544961:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307529856:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761315064.548401:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315064.548411:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761315064.548415:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761315064.548428:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3307 00000100:00080000:1.0:1761315064.548433:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1761315064.548439:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761315064.548452:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761315064.548470:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000163e3801 x1846869307530048/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761315064.548509:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315064.548512:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761315064.548667:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307529920:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315064.548771:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761315064.548777:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307529984:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315064.552669:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307530048:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761315064.558143:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307529984:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315064.558167:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307529920:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315064.559232:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307529856:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761315064.559689:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761315064.559695:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761315064.559701:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307530048:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315064.800812:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315064.864930:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315065.824787:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315065.889089:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315066.849247:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315066.914960:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315067.873348:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315067.937402:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315068.897562:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315068.961516:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761315069.666013:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315069.666023:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761315069.666027:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761315069.666040:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3312 00000100:00080000:3.0:1761315069.666044:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761315069.666051:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761315069.666063:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761315069.666101:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ddd3c50a x1846869307530304/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761315069.666131:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315069.666134:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761315069.666187:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307530176:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315069.666293:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1761315069.666299:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307530112:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315069.666320:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307530240:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315069.666436:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307530304:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761315069.670585:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307530240:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315069.670609:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307530176:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315069.670671:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307530112:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761315069.671607:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761315069.671614:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761315069.671622:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307530304:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315069.921447:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315069.984940:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315070.945915:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315071.010227:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315071.968791:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315072.033615:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315072.992831:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315073.056977:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315074.017412:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315074.081920:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761315074.785114:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315074.785121:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761315074.785125:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761315074.785136:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3317 00000100:00080000:3.0:1761315074.785139:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761315074.785145:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761315074.785154:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761315074.785170:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f133bb2e x1846869307530560/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761315074.785209:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315074.785213:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761315074.785240:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307530432:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315074.785333:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1761315074.785340:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307530368:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315074.785364:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307530496:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315074.786895:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f133bb2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307530560:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761315074.793173:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307530496:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315074.793193:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307530432:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315074.794163:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307530368:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761315074.795068:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761315074.795081:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761315074.795092:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f133bb2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307530560:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315075.040795:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315075.111726:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315076.065183:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315076.130713:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315077.088833:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315077.153125:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315078.112794:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315078.188871:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315079.136806:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315079.201219:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761315079.904999:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315079.905009:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761315079.905013:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761315079.905027:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3322 00000100:00080000:0.0:1761315079.905032:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761315079.905039:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761315079.905052:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761315079.905072:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a24c5c6a x1846869307530816/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761315079.905099:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315079.905104:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761315079.905151:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307530688:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761315079.905268:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1761315079.905276:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307530624:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761315079.905302:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307530752:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761315079.905401:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307530816:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761315079.909513:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307530688:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761315079.912581:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307530624:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761315079.913414:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307530752:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761315079.913781:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761315079.913786:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761315079.913791:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307530816:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761315080.160776:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315080.224849:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315081.184798:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315081.248857:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315082.208784:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315082.273604:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315083.234557:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315083.297584:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315084.258853:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315084.320880:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761315085.025355:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315085.025372:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761315085.025376:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761315085.025390:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3327 00000100:00080000:0.0:1761315085.025394:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761315085.025401:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761315085.025414:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761315085.025432:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000057b18867 x1846869307531072/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761315085.025458:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315085.025461:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761315085.025494:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307530944:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761315085.025617:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1761315085.025624:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307530880:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761315085.025683:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307531008:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761315085.025874:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307531072:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761315085.031108:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307530880:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761315085.031137:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307531008:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761315085.031148:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307530944:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761315085.031599:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761315085.031605:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761315085.031611:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307531072:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761315085.285591:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315085.346933:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315086.304794:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315086.370692:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315087.328946:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315087.394703:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315088.352946:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315088.419086:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315089.376823:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315089.441292:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761315090.145075:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315090.145083:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761315090.145087:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761315090.145101:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3333 00000100:00080000:2.0:1761315090.145104:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761315090.145112:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761315090.145127:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761315090.145145:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ba5c8e77 x1846869307531328/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761315090.145170:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315090.145173:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761315090.145196:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039978a4e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307531200:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315090.145379:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1761315090.145386:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e3466f5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307531136:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315090.145412:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000015f165e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307531264:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315090.145507:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ba5c8e77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307531328:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315090.149752:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039978a4e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307531200:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315090.151487:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e3466f5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307531136:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315090.153036:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000015f165e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307531264:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761315090.154134:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761315090.154141:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761315090.154147:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ba5c8e77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307531328:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315090.401385:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315090.466439:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315091.424840:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315091.488908:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315092.449244:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315092.515434:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315093.473119:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315093.536928:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315094.496777:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315094.562329:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761315095.264954:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315095.264964:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761315095.264967:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761315095.264980:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3338 00000100:00100000:1.0:1761315095.264981:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307531392:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761315095.264985:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761315095.264993:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761315095.265004:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761315095.265021:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008e59d025 x1846869307531584/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761315095.265049:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315095.265052:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761315095.265068:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307531456:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761315095.265075:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307531520:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761315095.265102:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307531584:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761315095.270572:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307531456:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761315095.270776:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761315095.270782:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761315095.270788:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307531584:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761315095.270881:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307531392:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761315095.270895:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307531520:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761315095.520783:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315095.585521:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315096.544763:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315096.609705:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315097.568748:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315097.633563:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315098.592777:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315098.656875:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315099.617336:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315099.681979:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761315100.384919:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315100.384927:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761315100.384931:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761315100.384944:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3343 00000100:00080000:2.0:1761315100.384948:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761315100.384954:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761315100.384964:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761315100.384981:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000001426be00 x1846869307531840/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1761315100.385000:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000015f165e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307531648:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761315100.385028:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315100.385032:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761315100.385062:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001426be00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307531840:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761315100.385068:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e3466f5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307531712:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761315100.385542:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039978a4e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307531776:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761315100.389380:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000015f165e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307531648:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761315100.390526:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e3466f5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307531712:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761315100.392543:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039978a4e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307531776:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761315100.394800:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761315100.394846:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761315100.394857:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001426be00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307531840:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761315100.642068:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315100.705321:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315101.665005:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315101.730187:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315102.689447:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315102.752885:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315103.712873:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315103.776900:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315104.736792:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315104.800860:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761315105.504968:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315105.504976:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761315105.504979:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761315105.504990:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3348 00000100:00080000:2.0:1761315105.504993:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761315105.504999:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761315105.505009:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761315105.505026:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ee08055d x1846869307532096/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761315105.505056:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315105.505060:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761315105.505163:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ee08055d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307532096:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761315105.505334:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e3466f5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307531904:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761315105.505368:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e9376149 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307532032:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761315105.505396:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761315105.505402:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eca34e1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307531968:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761315105.517218:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761315105.517233:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761315105.517246:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ee08055d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307532096:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761315105.518374:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eca34e1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307531968:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761315105.518393:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e9376149 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307532032:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761315105.518685:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e3466f5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307531904:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761315105.764234:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315105.825785:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315106.784797:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315106.848976:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315107.808810:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315107.874355:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315108.833409:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315108.896920:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315109.857275:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315109.920946:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761315110.625495:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315110.625503:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761315110.625507:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761315110.625518:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3353 00000100:00080000:2.0:1761315110.625522:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761315110.625552:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761315110.625566:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761315110.625585:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b8c05a37 x1846869307532352/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761315110.625611:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1761315110.625612:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ae09c97d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307532160:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761315110.625615:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761315110.625682:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307532352:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315110.625709:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002292ab57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307532224:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761315110.629384:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761315110.629402:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315110.629415:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307532352:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315110.629482:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002292ab57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307532224:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315110.632828:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004592c0a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307532288:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315110.635331:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ae09c97d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307532160:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315110.641776:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004592c0a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307532288:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315110.881095:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315110.948721:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315111.904796:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315111.968962:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315112.930178:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315112.993004:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315113.954022:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315114.017434:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315114.976810:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315115.041273:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315115.744855:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002292ab57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307532416:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761315115.747320:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315115.747328:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761315115.747331:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761315115.747342:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3358 00000100:00080000:2.0:1761315115.747345:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761315115.747350:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761315115.747362:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761315115.747379:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f220504f x1846869307532608/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761315115.747404:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315115.747408:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761315115.747413:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b2298797 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307532544:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761315115.747456:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307532480:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761315115.749464:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f220504f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307532608:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761315115.755334:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307532480:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761315115.755565:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b2298797 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307532544:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761315115.755577:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002292ab57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307532416:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761315115.756954:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761315115.756964:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761315115.756973:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f220504f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307532608:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761315116.001949:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315116.064911:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315117.024797:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315117.088926:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315118.049127:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315118.116178:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315119.072802:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315119.138306:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315120.096946:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315120.162735:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761315120.865192:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315120.865200:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761315120.865204:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761315120.865216:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3363 00000100:00080000:2.0:1761315120.865220:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761315120.865227:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761315120.865236:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761315120.865253:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b4eac89a x1846869307532864/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1761315120.865270:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307532672:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761315120.865279:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315120.865285:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761315120.865338:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000007136630 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307532736:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761315120.865339:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b4eac89a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307532864:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761315120.865420:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aea3302e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307532800:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761315120.869194:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aea3302e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307532800:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761315120.870739:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000007136630 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307532736:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761315120.870870:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761315120.870879:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761315120.870887:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b4eac89a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307532864:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761315120.870979:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307532672:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761315121.121021:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315121.185679:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315122.144792:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315122.209622:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315123.168790:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315123.234347:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315124.192776:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315124.257675:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315125.227088:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315125.280817:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761315125.990024:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315125.990031:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761315125.990035:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761315125.990045:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3368 00000100:00080000:3.0:1761315125.990048:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761315125.990055:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761315125.990065:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761315125.990082:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a183f7ef x1846869307533120/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761315125.990117:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315125.990121:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761315125.990154:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307532992:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315125.990169:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307533120:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761315125.990267:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1761315125.990275:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307532928:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315125.990300:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307533056:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315125.999728:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307532928:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315125.999759:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307532992:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315126.000362:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307533056:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761315126.000401:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761315126.000411:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761315126.000431:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307533120:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761315126.240976:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315126.307559:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315127.265696:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315127.329208:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315128.289158:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315128.353273:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315129.312801:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315129.380622:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315130.339074:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315130.401345:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315131.106222:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307533184:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761315131.107881:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315131.107887:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761315131.107890:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761315131.107899:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3374 00000100:00080000:1.0:1761315131.107902:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1761315131.107922:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761315131.107931:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761315131.107946:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ed2f9bca x1846869307533376/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761315131.107975:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315131.107978:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761315131.108056:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307533376:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761315131.108846:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307533248:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761315131.108885:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761315131.108890:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307533312:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761315131.112948:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761315131.112960:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315131.112969:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307533376:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761315131.113777:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307533184:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761315131.113830:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307533248:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761315131.113841:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307533312:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761315131.361534:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315131.424854:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315132.385511:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315132.449737:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315133.408765:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315133.475056:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315134.433858:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315134.498777:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315135.457398:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315135.521073:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761315136.224898:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315136.224906:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761315136.224910:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761315136.224923:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3379 00000100:00080000:0.0:1761315136.224928:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761315136.224935:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761315136.224947:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761315136.224984:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000fe4020c4 x1846869307533632/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761315136.225014:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315136.225017:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761315136.225102:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307533504:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315136.225214:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1761315136.225219:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307533440:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315136.225248:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307533568:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315136.226208:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307533632:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761315136.229080:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307533440:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315136.229113:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307533568:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315136.229127:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307533504:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761315136.235516:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761315136.235531:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315136.235544:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307533632:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761315136.481616:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315136.546806:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315137.505678:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315137.569356:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315138.528808:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315138.594069:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315139.553076:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315139.617577:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315140.576766:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315140.643713:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761315141.346785:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315141.346791:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761315141.346794:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761315141.346804:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3384 00000100:00080000:0.0:1761315141.346807:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761315141.346813:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761315141.346824:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761315141.346839:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000094820351 x1846869307533888/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761315141.346864:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315141.346867:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761315141.346955:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307533696:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315141.347056:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307533824:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315141.347087:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761315141.347094:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307533760:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315141.349072:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307533888:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761315141.352164:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307533760:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315141.352182:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307533696:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315141.354468:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307533824:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761315141.360633:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761315141.360683:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315141.360694:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307533888:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761315141.601057:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315141.666553:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315142.624807:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315142.691047:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315143.651911:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315143.713037:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315144.673339:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315144.737405:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315145.696794:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315145.762346:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761315146.464862:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315146.464869:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761315146.464871:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761315146.464880:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3389 00000100:00080000:1.0:1761315146.464884:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1761315146.464889:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761315146.464898:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761315146.464913:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ed2f9bca x1846869307534144/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761315146.464937:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315146.464940:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761315146.464960:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307534016:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761315146.465038:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1761315146.465044:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307533952:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761315146.465067:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307534080:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761315146.465150:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307534144:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761315146.468978:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307533952:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761315146.469171:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307534016:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761315146.471826:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307534080:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761315146.472357:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761315146.472363:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315146.472369:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307534144:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761315146.720854:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315146.784859:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315147.744783:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315147.809022:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315148.768818:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315148.832873:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315149.792897:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315149.856848:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315150.816797:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315150.881705:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315151.585176:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307534208:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315151.585556:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761315151.585563:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307534272:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761315151.585763:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315151.585777:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761315151.585780:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761315151.585785:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3394 00000100:00080000:0.0:1761315151.585788:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761315151.585794:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761315151.585803:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761315151.585819:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000005b900d12 x1846869307534400/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761315151.585830:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315151.585833:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761315151.585847:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761315151.585856:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307534336:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315151.586368:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307534400:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761315151.589451:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307534208:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315151.594305:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307534272:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761315151.598479:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:1.0:1761315151.598485:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307534336:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761315151.598488:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315151.598492:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307534400:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761315151.840781:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315151.904866:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315152.864806:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315152.930275:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315153.889879:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315153.954667:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315154.916489:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315154.976850:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315155.936734:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315156.001798:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761315156.704832:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315156.704840:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761315156.704843:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761315156.704854:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3399 00000100:00080000:0.0:1761315156.704857:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761315156.704862:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761315156.704871:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761315156.704901:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000546b1bfe x1846869307534656/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761315156.704923:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315156.704926:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761315156.704952:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307534528:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315156.705042:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1761315156.705048:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307534464:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315156.705069:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307534592:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315156.705159:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307534656:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761315156.710041:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307534464:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761315156.714020:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761315156.714030:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315156.714039:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307534656:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761315156.714136:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307534592:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315156.714148:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307534528:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315156.960774:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315157.033895:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315157.984770:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315158.049263:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315159.008774:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315159.073024:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315160.032776:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315160.096999:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315161.056788:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315161.120839:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761315161.825405:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315161.825414:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761315161.825417:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761315161.825428:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3404 00000100:00080000:1.0:1761315161.825432:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1761315161.825439:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761315161.825449:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761315161.825466:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a1719d14 x1846869307534912/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761315161.825491:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315161.825495:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761315161.825525:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307534784:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761315161.825543:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307534912:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761315161.825625:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1761315161.825633:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307534720:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761315161.825696:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307534848:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761315161.830895:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:0.0:1761315161.830959:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307534720:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761315161.830976:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307534848:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761315161.830990:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307534784:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761315161.830994:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315161.830998:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307534912:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761315162.081052:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315162.145162:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315163.104961:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315163.168901:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315164.129046:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315164.193008:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315165.153221:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315165.217129:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315166.176873:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315166.240909:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761315166.946029:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315166.946037:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761315166.946040:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761315166.946052:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3409 00000100:00080000:0.0:1761315166.946055:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00100000:1.0:1761315166.946061:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307534976:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761315166.946063:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761315166.946077:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761315166.946110:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000034a6cd79 x1846869307535168/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761315166.946142:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315166.946146:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761315166.946168:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307535040:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315166.946177:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307535104:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315166.946294:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307535168:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761315166.951403:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307534976:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315166.951405:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307535040:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315166.951429:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307535104:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761315166.954478:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761315166.954488:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315166.954498:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307535168:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761315167.200795:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315167.265561:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315168.227788:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315168.288964:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315169.249758:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315169.313454:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315170.272765:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315170.337015:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315171.296800:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315171.365585:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761315172.064909:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315172.064916:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761315172.064919:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761315172.064929:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3415 00000100:00080000:1.0:1761315172.064932:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1761315172.064939:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761315172.064949:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761315172.064965:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a15ef099 x1846869307535424/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761315172.064990:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315172.064994:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761315172.065020:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307535296:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761315172.065130:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1761315172.065137:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307535232:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761315172.065163:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307535360:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761315172.065452:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307535424:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761315172.070875:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307535232:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761315172.070899:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307535296:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761315172.072120:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761315172.072126:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315172.072132:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307535424:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761315172.072716:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307535360:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761315172.321248:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315172.385186:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315173.346321:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315173.409809:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315174.369299:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315174.432824:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315175.392764:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315175.456952:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315176.416983:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315176.482325:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761315177.184883:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315177.184891:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761315177.184895:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761315177.184907:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3420 00000100:00100000:2.0:1761315177.184928:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307535488:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761315177.184928:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761315177.184938:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761315177.184951:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761315177.184972:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e6d9e11e x1846869307535680/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761315177.185005:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315177.185008:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761315177.185032:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307535552:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761315177.185043:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307535616:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761315177.185198:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307535680:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761315177.187964:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307535552:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761315177.191237:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307535616:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761315177.195540:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761315177.195550:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761315177.195559:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307535680:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761315177.195656:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307535488:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761315177.442299:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315177.504842:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315178.465081:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315178.529682:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315179.490514:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315179.557280:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315180.512819:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315180.576897:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315181.536794:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315181.600878:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761315182.304904:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315182.304911:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761315182.304914:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761315182.304924:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3425 00000100:00080000:0.0:1761315182.304927:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761315182.304933:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761315182.304941:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761315182.304956:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e00b5e9d x1846869307535936/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761315182.304983:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315182.304986:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761315182.305010:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307535808:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315182.305129:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1761315182.305137:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307535744:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315182.305158:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307535872:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315182.305670:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307535936:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761315182.310005:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307535744:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315182.310037:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307535872:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315182.310046:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307535808:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761315182.312557:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761315182.312565:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315182.312573:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307535936:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761315182.562113:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315182.625672:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315183.585191:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315183.650315:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315184.608769:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315184.675038:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315185.632777:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315185.697607:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315186.656812:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315186.723253:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761315187.425521:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315187.425527:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761315187.425530:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761315187.425540:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3430 00000100:00080000:2.0:1761315187.425543:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761315187.425549:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761315187.425558:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761315187.425574:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b4eac89a x1846869307536192/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761315187.425601:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315187.425605:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761315187.425629:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000039eebc2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307536064:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315187.425709:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000007136630 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307536000:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315187.425803:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b4eac89a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307536192:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761315187.427702:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307536128:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761315187.440047:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761315187.440056:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761315187.440065:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b4eac89a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307536192:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315187.440171:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000039eebc2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307536064:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315187.441212:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000007136630 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307536000:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315187.441238:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307536128:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315187.681038:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315187.749527:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315188.705078:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315188.769063:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315189.728768:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315189.792893:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315190.752745:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315190.817563:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315191.777048:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315191.842875:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761315192.544985:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315192.544992:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761315192.544995:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761315192.545004:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3435 00000100:00080000:0.0:1761315192.545007:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761315192.545012:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761315192.545021:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761315192.545035:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000bcaee194 x1846869307536448/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761315192.545058:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315192.545061:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761315192.547987:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307536448:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761315192.548441:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307536256:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315192.548481:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307536384:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315192.548505:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761315192.548512:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307536320:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315192.554864:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307536320:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315192.554888:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307536256:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315192.554899:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307536384:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761315192.556822:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761315192.556837:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315192.556850:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307536448:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761315192.801002:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315192.867580:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315193.829482:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315193.888907:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315194.848983:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315194.913606:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315195.873125:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315195.936845:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315196.898683:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315196.960842:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761315197.665615:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315197.665622:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761315197.665626:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761315197.665668:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3440 00000100:00080000:1.0:1761315197.665672:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761315197.665678:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761315197.665686:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761315197.665703:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000009cec3faa x1846869307536704/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761315197.665725:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315197.665728:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761315197.665748:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307536576:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315197.665789:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307536704:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761315197.665832:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1761315197.665841:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307536512:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315197.665864:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307536640:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761315197.670580:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761315197.670591:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315197.670601:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307536704:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761315197.681026:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307536512:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315197.681049:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307536640:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315197.681060:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307536576:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315197.920781:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315197.984852:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315198.947025:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315199.011347:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315199.969164:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315200.032907:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315200.992791:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315201.057948:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315202.016810:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315202.080812:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761315202.787842:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315202.787852:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761315202.787856:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761315202.787869:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3445 00000100:00080000:3.0:1761315202.787873:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761315202.787892:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761315202.787904:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761315202.787923:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006e93398e x1846869307536960/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761315202.787960:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315202.787963:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761315202.788000:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cc6dd8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307536832:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761315202.788139:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1761315202.788146:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307536768:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761315202.788171:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f597cda6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307536896:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761315202.790506:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307536960:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761315202.795357:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307536768:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761315202.795389:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f597cda6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307536896:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761315202.795402:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc6dd8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307536832:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761315202.796379:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761315202.796390:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761315202.796417:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307536960:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761315203.040970:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315203.111133:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315204.073480:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315204.129577:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315205.089258:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315205.155035:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315206.112796:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315206.177146:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315207.136788:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315207.200897:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761315207.905963:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315207.905974:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761315207.905978:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761315207.905991:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3450 00000100:00080000:3.0:1761315207.905996:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761315207.906003:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761315207.906016:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761315207.906034:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f364ec74 x1846869307537216/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761315207.906063:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315207.906066:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761315207.906429:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cc6dd8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307537024:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315207.907001:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f364ec74 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307537216:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315207.909199:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307537152:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315207.910328:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f597cda6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307537088:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315207.918592:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f597cda6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307537088:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315207.918742:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc6dd8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307537024:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315207.918755:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307537152:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761315207.918859:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761315207.918865:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761315207.918870:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f364ec74 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307537216:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315208.160807:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315208.226130:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315209.186106:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315209.249907:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315210.208821:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315210.272885:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315211.232803:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315211.297544:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315212.256956:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315212.325779:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761315213.024999:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315213.025010:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761315213.025014:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761315213.025026:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3455 00000100:00080000:3.0:1761315213.025031:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761315213.025038:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761315213.025071:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761315213.025092:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000009911ad3 x1846869307537472/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761315213.025119:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315213.025123:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761315213.025307:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000009911ad3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307537472:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315213.025733:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f597cda6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307537280:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315213.025907:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000101f852a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307537408:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315213.025944:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761315213.025951:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307537344:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:02020000:0.0:1761315213.027474:0:2262:0:(client.c:1361:ptlrpc_check_status()) 11-0: lustre-OST0000-osc-ffff951bc4d8c800: operation ost_connect to node 192.168.203.151@tcp failed: rc = -19 00000100:00080000:0.0:1761315213.027489:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761315213.027494:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761315213.027499:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000009911ad3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307537472:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315213.027588:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307537344:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315213.027599:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000101f852a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307537408:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315213.029310:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f597cda6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307537280:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315213.280965:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315213.345389:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315214.304793:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315214.368881:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315215.329000:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315215.393583:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315216.353038:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315216.418589:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315217.376769:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315217.442104:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761315218.145495:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315218.145505:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761315218.145509:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761315218.145522:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3461 00000100:00080000:1.0:1761315218.145526:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761315218.145533:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761315218.145545:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761315218.145565:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000009cec3faa x1846869307537728/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761315218.145719:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307537728:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00080000:1.0:1761315218.145876:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315218.145881:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761315218.146303:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307537536:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761315218.146355:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307537664:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761315218.146385:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761315218.146392:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307537600:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761315218.152127:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761315218.152145:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315218.152157:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307537728:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761315218.154701:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307537600:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761315218.154747:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307537664:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761315218.156712:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307537536:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761315218.400843:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315218.468036:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315219.424969:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315219.492419:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315220.449417:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315220.513974:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315221.472821:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315221.539527:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315222.497430:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315222.562783:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761315223.265986:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315223.265995:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761315223.265999:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761315223.266011:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3466 00000100:00080000:2.0:1761315223.266015:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761315223.266022:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761315223.266033:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761315223.266050:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000002292ab57 x1846869307537984/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761315223.266075:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315223.266079:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761315223.266113:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aea3302e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307537856:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315223.266127:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002292ab57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307537984:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315223.266298:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1761315223.266307:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000039eebc2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307537792:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315223.266332:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f220504f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307537920:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315223.268314:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aea3302e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307537856:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315223.275986:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000039eebc2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307537792:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315223.276017:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f220504f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307537920:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761315223.276145:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761315223.276151:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761315223.276156:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002292ab57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307537984:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315223.521774:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315223.591298:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315224.544765:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315224.610989:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315225.568774:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315225.632907:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315226.593213:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315226.658370:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315227.616801:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315227.680915:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761315228.385369:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315228.385376:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761315228.385380:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761315228.385391:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3471 00000100:00080000:2.0:1761315228.385395:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761315228.385401:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761315228.385412:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761315228.385431:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b2298797 x1846869307538240/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1761315228.385458:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f220504f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307538048:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315228.385512:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b2298797 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307538240:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761315228.385593:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aea3302e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307538176:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315228.385691:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761315228.385714:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000039eebc2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307538112:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761315228.386053:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315228.386058:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00080000:0.0:1761315228.390173:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761315228.390183:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761315228.390192:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b2298797 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307538240:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761315228.390360:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f220504f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307538048:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315228.390375:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aea3302e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307538176:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315228.390556:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000039eebc2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307538112:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315228.641833:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315228.704998:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315229.664832:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315229.729954:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315230.688760:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315230.753370:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315231.712816:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315231.776887:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315232.739074:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315232.803499:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761315233.504975:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315233.504983:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761315233.504987:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761315233.504997:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3476 00000100:00080000:3.0:1761315233.505002:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761315233.505008:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761315233.505017:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761315233.505032:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000041c8b6cb x1846869307538496/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761315233.505059:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315233.505063:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761315233.505085:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e58cc722 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307538368:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761315233.505268:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1761315233.505275:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007673f690 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307538304:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761315233.505302:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d32bed84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307538432:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761315233.506240:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000041c8b6cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307538496:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00080000:0.0:1761315233.509302:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761315233.509314:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761315233.509326:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000041c8b6cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307538496:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761315233.510741:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007673f690 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307538304:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761315233.510767:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e58cc722 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307538368:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761315233.513339:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d32bed84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307538432:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761315233.760825:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315233.824880:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315234.784759:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315234.852394:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315235.808780:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315235.877755:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315236.832770:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315236.898246:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315237.856743:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315237.921177:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761315238.625919:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1761315238.626016:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004592c0a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307538560:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315238.626110:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e3466f5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307538688:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315238.626143:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761315238.626148:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ae09c97d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307538624:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761315238.626232:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761315238.626236:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761315238.626242:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3481 00000100:00080000:2.0:1761315238.626244:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761315238.626250:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761315238.626261:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761315238.626277:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e9376149 x1846869307538752/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761315238.626362:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e9376149 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307538752:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00080000:2.0:1761315238.629477:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315238.629488:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00080000:0.0:1761315238.631451:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761315238.631461:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761315238.631469:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e9376149 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307538752:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315238.631564:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ae09c97d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307538624:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315238.631577:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004592c0a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307538560:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315238.631586:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e3466f5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307538688:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315238.880764:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315238.945106:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315239.904759:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315239.968855:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315240.928857:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315240.992973:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315241.952778:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315242.018803:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315242.976809:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315243.043218:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761315243.744853:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315243.744860:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761315243.744864:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761315243.744875:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3486 00000100:00080000:2.0:1761315243.744879:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761315243.744885:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761315243.744896:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761315243.744911:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000eca34e1c x1846869307539008/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761315243.744941:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315243.744945:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761315243.744971:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004592c0a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307538880:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315243.745075:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1761315243.745081:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e3466f5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307538816:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315243.745109:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ae09c97d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307538944:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315243.745130:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eca34e1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307539008:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315243.751274:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ae09c97d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307538944:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315243.752464:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e3466f5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307538816:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315243.752506:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004592c0a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307538880:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761315243.752924:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761315243.752929:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761315243.752935:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eca34e1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307539008:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315244.001675:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315244.065538:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315245.025252:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315245.088921:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315246.052730:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315246.113298:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315247.072802:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315247.137210:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315248.096774:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315248.161078:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761315248.865693:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315248.865705:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761315248.865710:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761315248.865726:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3491 00000100:00080000:2.0:1761315248.865731:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761315248.865739:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000100:00100000:3.0:1761315248.865746:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004592c0a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307539072:192.168.203.151@tcp:400:kworker/u8:1.0 00000008:00100000:2.0:1761315248.865757:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761315248.865782:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ee08055d x1846869307539264/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761315248.865800:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315248.865804:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761315248.865830:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e3466f5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307539136:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315248.865873:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ae09c97d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307539200:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315248.866489:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ee08055d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307539264:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761315248.868857:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ae09c97d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307539200:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315248.870533:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e3466f5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307539136:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315248.873219:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004592c0a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307539072:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761315248.873455:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761315248.873462:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315248.873468:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ee08055d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307539264:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315249.120833:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315249.184929:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315250.145739:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315250.209808:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315251.168839:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315251.233280:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315252.192800:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315252.256932:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315253.216806:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315253.283041:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315253.985919:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307539328:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761315253.987177:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315253.987185:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761315253.987189:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761315253.987200:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3496 00000100:00080000:1.0:1761315253.987204:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761315253.987211:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761315253.987225:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761315253.987245:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000163e3801 x1846869307539520/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761315253.987310:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307539520:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761315253.989572:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307539392:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761315253.989759:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315253.989765:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761315253.989960:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307539456:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761315253.990941:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307539328:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761315253.996439:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761315253.996454:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315253.996468:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307539520:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761315253.999745:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307539456:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761315254.001313:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307539392:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761315254.242186:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315254.304927:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315255.265227:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315255.331790:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315256.288793:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315256.352943:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315257.314414:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315257.378197:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315258.337685:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315258.402062:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761315259.105451:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315259.105459:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761315259.105461:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761315259.105472:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3502 00000100:00080000:2.0:1761315259.105476:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761315259.105491:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761315259.105501:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761315259.105516:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000039978a4e x1846869307539776/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761315259.105543:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315259.105555:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761315259.105816:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039978a4e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307539776:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761315259.106387:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e3466f5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307539648:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315259.107365:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ee08055d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307539584:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315259.107434:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001426be00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307539712:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761315259.110976:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761315259.110987:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761315259.110996:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039978a4e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307539776:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761315259.112184:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e3466f5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307539648:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315259.113669:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ee08055d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307539584:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315259.113696:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001426be00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307539712:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315259.360774:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315259.424949:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315260.384776:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315260.451419:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315261.409897:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315261.472868:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315262.432767:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315262.496867:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315263.456748:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315263.520871:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761315264.225001:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315264.225008:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761315264.225011:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761315264.225021:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3507 00000100:00080000:2.0:1761315264.225024:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761315264.225029:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761315264.225039:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761315264.225066:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ba5c8e77 x1846869307540032/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761315264.225092:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315264.225096:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761315264.225105:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001426be00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307539840:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315264.225121:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ee08055d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307539904:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315264.225138:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ba5c8e77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307540032:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761315264.225381:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000015f165e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307539968:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315264.229780:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ee08055d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307539904:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315264.229837:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000015f165e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307539968:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761315264.229908:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761315264.229914:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761315264.229919:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ba5c8e77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307540032:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761315264.230230:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001426be00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307539840:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315264.480944:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315264.545042:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315265.504781:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315265.568994:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315266.528815:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315266.592867:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315267.552798:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315267.616927:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315268.576811:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315268.642362:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315269.346346:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307540096:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761315269.346450:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315269.346456:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761315269.346460:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761315269.346466:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3512 00000100:00080000:1.0:1761315269.346470:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761315269.346475:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000100:00100000:0.0:1761315269.346479:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307540224:192.168.203.151@tcp:400:kworker/u8:2.0 00000008:00100000:1.0:1761315269.346488:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761315269.346505:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761315269.346506:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000540c3ce6 x1846869307540288/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761315269.346509:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307540160:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761315269.346534:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315269.346537:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761315269.346585:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307540288:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761315269.361465:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307540160:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761315269.361487:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307540224:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761315269.361687:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761315269.361694:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315269.361699:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307540288:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761315269.363400:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307540096:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761315269.600791:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315269.665975:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315270.624859:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315270.688864:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315271.648769:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315271.712892:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315272.672759:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315272.737945:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315273.696998:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315273.760984:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761315274.465087:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315274.465095:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761315274.465098:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761315274.465110:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3517 00000100:00080000:2.0:1761315274.465114:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00100000:3.0:1761315274.465116:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ee08055d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307540352:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761315274.465123:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761315274.465136:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761315274.465154:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000011b5c99f x1846869307540544/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1761315274.465220:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008a641e2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307540480:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315274.465235:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000011b5c99f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307540544:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761315274.465256:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761315274.465265:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b3aaddc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307540416:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761315274.465747:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315274.465753:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00080000:0.0:1761315274.469146:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761315274.469160:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761315274.469170:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000011b5c99f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307540544:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761315274.469265:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b3aaddc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307540416:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315274.469277:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ee08055d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307540352:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315274.469307:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008a641e2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307540480:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315274.720874:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315274.787665:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315275.744722:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315275.809062:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315276.768865:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315276.833027:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315277.794267:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315277.857992:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315278.816801:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315278.880930:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761315279.585018:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315279.585028:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761315279.585032:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761315279.585045:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3522 00000100:00080000:1.0:1761315279.585050:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761315279.585056:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761315279.585069:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761315279.585121:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a15ef099 x1846869307540800/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761315279.585149:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315279.585153:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761315279.585220:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307540800:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761315279.585419:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307540608:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761315279.585468:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307540736:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761315279.585497:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761315279.585503:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307540672:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761315279.591914:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761315279.591930:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315279.591943:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307540800:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761315279.592209:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307540736:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761315279.592234:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307540672:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761315279.592246:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307540608:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761315279.842243:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315279.905348:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315280.864827:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315280.929151:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315281.888807:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315281.952943:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315282.912821:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315282.976911:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315283.936812:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315284.001257:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761315284.712973:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315284.712982:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761315284.712986:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761315284.712998:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3527 00000100:00080000:3.0:1761315284.713002:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761315284.713009:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761315284.713021:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761315284.713042:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000041c8b6cb x1846869307541056/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761315284.713070:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315284.713073:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761315284.713109:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e58cc722 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307540928:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761315284.713307:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000041c8b6cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307541056:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761315284.717522:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d32bed84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307540864:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761315284.721775:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007673f690 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307540992:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761315284.724015:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e58cc722 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307540928:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761315284.725374:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761315284.725395:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761315284.725404:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000041c8b6cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307541056:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761315284.726877:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d32bed84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307540864:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761315284.728044:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007673f690 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307540992:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761315284.961720:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315285.025265:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315285.987874:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315286.049484:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315287.010714:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315287.072850:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315288.033654:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315288.097336:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315289.059249:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315289.126131:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761315289.825851:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315289.825860:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761315289.825864:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761315289.825878:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3532 00000100:00080000:1.0:1761315289.825883:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761315289.825891:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000100:00100000:0.0:1761315289.825900:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307541120:192.168.203.151@tcp:400:kworker/u8:2.0 00000008:00100000:1.0:1761315289.825902:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761315289.825923:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000009cec3faa x1846869307541312/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761315289.825953:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315289.825957:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761315289.826005:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307541184:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761315289.826011:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307541248:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761315289.827066:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307541312:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761315289.830336:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307541120:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761315289.830388:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307541248:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761315289.830560:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307541184:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761315289.834998:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761315289.835013:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315289.835026:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307541312:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761315290.081028:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315290.145410:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315291.105708:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315291.169063:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315292.128789:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315292.194426:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315293.153039:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315293.217325:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315294.176977:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315294.241480:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761315294.944917:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315294.944928:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761315294.944932:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761315294.944944:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3537 00000100:00100000:0.0:1761315294.944945:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307541376:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761315294.944948:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761315294.944955:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761315294.944969:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761315294.944988:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000009cec3faa x1846869307541568/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761315294.945014:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315294.945017:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761315294.945041:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307541440:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761315294.945057:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307541504:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761315294.945455:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307541568:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761315294.951129:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307541376:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761315294.951152:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307541504:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761315294.951277:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761315294.951282:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315294.951287:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307541568:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761315294.951372:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307541440:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761315295.201255:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315295.265674:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315296.227707:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315296.289699:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315297.248814:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315297.314229:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315298.272807:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315298.336875:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315299.296909:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315299.361705:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315300.064921:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00080000:1.0:1761315300.064925:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315300.064934:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761315300.064938:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761315300.064953:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3543 00000100:00100000:0.0:1761315300.064954:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307541696:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761315300.064976:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761315300.064984:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761315300.064999:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761315300.065020:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ed2f9bca x1846869307541824/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761315300.065049:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315300.065053:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761315300.065083:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307541632:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761315300.065109:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761315300.065124:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307541760:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761315300.065405:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307541824:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761315300.072140:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307541632:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761315300.075352:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307541760:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761315300.075378:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307541696:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761315300.078371:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761315300.078386:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315300.078399:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307541824:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761315300.320797:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315300.385392:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315301.348667:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315301.411627:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315302.368746:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315302.432873:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315303.392761:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315303.458262:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315304.416793:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315304.480952:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761315305.185732:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315305.185740:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761315305.185744:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761315305.185755:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3548 00000100:00080000:0.0:1761315305.185758:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761315305.185765:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761315305.185775:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761315305.185792:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000005b900d12 x1846869307542080/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761315305.185819:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315305.185823:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761315305.185854:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307541952:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315305.185873:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307542080:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761315305.186358:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307541888:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315305.187721:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307542016:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761315305.205310:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761315305.205320:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315305.205330:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307542080:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761315305.205332:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307541952:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315305.205826:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307541888:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315305.205840:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307542016:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315305.440765:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315305.505301:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315306.464757:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315306.528881:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315307.492279:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315307.552872:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315308.512840:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315308.579705:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315309.536800:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315309.605399:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761315310.304937:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315310.304947:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761315310.304950:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761315310.304963:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3553 00000100:00080000:3.0:1761315310.304967:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761315310.304975:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761315310.304988:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761315310.305005:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000009911ad3 x1846869307542336/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761315310.305035:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315310.305039:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761315310.305155:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000009911ad3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307542336:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761315310.305340:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f597cda6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307542144:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761315310.305376:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307542272:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761315310.305400:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761315310.305406:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000101f852a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307542208:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761315310.309587:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761315310.309622:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761315310.309686:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000009911ad3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307542336:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761315310.311862:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307542272:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761315310.316880:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000101f852a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307542208:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761315310.326462:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f597cda6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307542144:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761315310.560817:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315310.626736:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315311.584790:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315311.648905:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315312.608809:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315312.681538:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315313.634846:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315313.697885:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315314.659947:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315314.721381:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761315315.424911:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315315.424917:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761315315.424920:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761315315.424930:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3558 00000100:00080000:1.0:1761315315.424933:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761315315.424939:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761315315.424949:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761315315.424963:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a15ef099 x1846869307542592/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761315315.424978:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307542400:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761315315.424991:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315315.424996:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761315315.425020:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307542464:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315315.425099:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307542592:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761315315.425115:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307542528:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315315.435867:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307542400:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315315.436250:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307542528:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315315.436328:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307542464:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761315315.438686:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761315315.438701:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315315.438713:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307542592:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315315.680785:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315315.744909:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315316.705120:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315316.770542:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315317.728812:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315317.792856:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315318.752801:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315318.818039:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315319.776842:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315319.840907:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315320.544918:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f364ec74 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307542656:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315320.545705:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307542720:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315320.546858:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cc6dd8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307542784:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761315320.548241:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315320.548249:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761315320.548252:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761315320.548261:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3563 00000100:00080000:3.0:1761315320.548264:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761315320.548270:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761315320.548282:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761315320.548306:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006e93398e x1846869307542848/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761315320.548331:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315320.548335:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761315320.548765:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307542848:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315320.550886:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f364ec74 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307542656:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761315320.555283:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761315320.555294:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761315320.555304:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307542848:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315320.558752:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc6dd8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307542784:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315320.558845:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307542720:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315320.803073:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315320.864904:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315321.824795:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315321.896948:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315322.848817:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315322.913422:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315323.873218:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315323.937595:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315324.896790:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315324.961856:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761315325.664837:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315325.664845:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761315325.664847:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761315325.664858:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3568 00000100:00080000:3.0:1761315325.664861:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761315325.664867:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761315325.664876:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761315325.664890:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000004c6aebc5 x1846869307543104/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761315325.664931:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315325.664934:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761315325.665030:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307542912:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315325.665118:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307543040:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315325.665152:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761315325.665159:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307542976:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315325.665272:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307543104:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761315325.668428:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307542976:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315325.668446:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307543040:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761315325.668607:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761315325.668612:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315325.668616:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307543104:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761315325.670828:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307542912:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315325.921136:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315325.985356:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315326.947456:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315327.009196:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315327.968753:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315328.033438:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315328.994022:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315329.060142:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315330.016765:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315330.080853:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761315330.787173:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315330.787180:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761315330.787183:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761315330.787194:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3573 00000100:00080000:3.0:1761315330.787198:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761315330.787204:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761315330.787213:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761315330.787227:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a183f7ef x1846869307543360/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761315330.787254:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315330.787257:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761315330.787283:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307543232:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315330.787398:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1761315330.787405:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307543168:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315330.787429:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307543296:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315330.787561:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307543360:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761315330.789867:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307543168:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761315330.792053:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761315330.792062:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761315330.792070:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307543360:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761315330.792318:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307543296:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315330.792331:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307543232:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315331.040765:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315331.104877:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315332.064821:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315332.128894:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315333.089743:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315333.157066:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315334.112783:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315334.176939:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315335.136779:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315335.201928:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761315335.905099:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315335.905110:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761315335.905114:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761315335.905130:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3578 00000100:00080000:3.0:1761315335.905134:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761315335.905141:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761315335.905154:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761315335.905173:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f133bb2e x1846869307543616/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761315335.905202:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315335.905205:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761315335.905239:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307543488:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315335.905386:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1761315335.905393:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307543424:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315335.905418:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307543552:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315335.905857:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f133bb2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307543616:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761315335.917021:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307543424:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315335.917054:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307543552:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315335.917066:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307543488:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761315335.917393:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761315335.917399:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761315335.917404:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f133bb2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307543616:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315336.160772:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315336.228166:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315337.184775:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315337.248797:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315338.208771:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315338.273437:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315339.232766:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315339.297366:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315340.256769:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315340.320879:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761315341.024887:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315341.024894:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761315341.024909:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761315341.024920:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3583 00000100:00080000:3.0:1761315341.024923:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761315341.024929:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761315341.024942:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761315341.024957:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ddd3c50a x1846869307543872/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761315341.024981:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315341.024984:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761315341.025013:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307543744:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761315341.025028:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307543872:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761315341.025143:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307543680:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761315341.025381:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307543808:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761315341.032448:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761315341.032459:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761315341.032468:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307543872:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761315341.037818:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307543744:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761315341.037885:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307543680:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761315341.037910:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307543808:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761315341.280775:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315341.347428:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315342.304727:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315342.369399:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315343.328764:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315343.392906:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315344.353113:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315344.421417:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315345.377192:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315345.440906:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761315346.144917:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315346.144927:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761315346.144931:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761315346.144944:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3589 00000100:00080000:3.0:1761315346.144948:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761315346.144955:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761315346.144967:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761315346.144986:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000039277a44 x1846869307544128/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761315346.145008:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315346.145012:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761315346.147979:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307544128:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315346.148188:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307543936:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315346.148227:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307544064:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315346.148249:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761315346.148255:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307544000:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315346.153506:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307543936:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761315346.154811:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761315346.154820:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761315346.154828:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307544128:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315346.155484:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307544000:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315346.155498:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307544064:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315346.402290:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315346.471206:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315347.424757:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315347.488900:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315348.448780:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315348.514476:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315349.472755:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315349.536908:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315350.496765:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315350.563706:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761315351.266772:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315351.266779:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761315351.266782:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761315351.266792:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3594 00000100:00080000:1.0:1761315351.266795:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761315351.266801:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761315351.266811:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761315351.266825:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a1719d14 x1846869307544384/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761315351.266860:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315351.266864:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761315351.266889:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307544256:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315351.266902:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307544384:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761315351.266978:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1761315351.266985:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307544192:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315351.267009:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307544320:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315351.272339:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307544256:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315351.274919:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307544192:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315351.274935:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307544320:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761315351.275745:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761315351.275756:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315351.275765:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307544384:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315351.520853:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315351.585228:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315352.544799:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315352.608889:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315353.568793:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315353.632965:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315354.592774:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315354.656965:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315355.616828:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315355.682883:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761315356.387341:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315356.387350:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761315356.387355:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761315356.387365:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3599 00000100:00080000:1.0:1761315356.387621:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761315356.387630:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761315356.387664:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761315356.387683:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ed2f9bca x1846869307544640/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761315356.387706:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315356.387709:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761315356.387782:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307544640:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761315356.388241:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307544448:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315356.388285:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307544576:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315356.388329:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761315356.388338:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307544512:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761315356.391473:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761315356.391489:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315356.391500:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307544640:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761315356.391856:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307544448:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315356.392533:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307544512:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315356.392547:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307544576:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315356.641900:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315356.706490:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315357.664989:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315357.729304:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315358.688828:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315358.757739:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315359.712792:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315359.777917:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315360.737238:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315360.802082:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315361.504901:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307544704:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761315361.506114:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315361.506120:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761315361.506123:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761315361.506130:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3604 00000100:00080000:0.0:1761315361.506134:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761315361.506139:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761315361.506149:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761315361.506163:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000084f3d917 x1846869307544896/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761315361.506186:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315361.506189:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761315361.506209:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307544832:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761315361.506223:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307544896:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761315361.506290:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307544768:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761315361.510762:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307544704:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761315361.512147:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761315361.512154:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315361.512162:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307544896:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761315361.512379:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307544832:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761315361.516212:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307544768:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761315361.761019:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315361.825209:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315362.784834:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315362.848858:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315363.809055:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315363.874835:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315364.833290:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315364.897364:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315365.858354:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315365.921067:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315366.626057:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307544960:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315366.626263:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761315366.626271:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307545024:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761315366.627502:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315366.627509:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761315366.627513:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761315366.627523:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3609 00000100:00080000:1.0:1761315366.627526:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761315366.627533:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761315366.627544:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761315366.627559:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a15ef099 x1846869307545152/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761315366.627582:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315366.627585:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761315366.627634:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307545152:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761315366.628149:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761315366.628163:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307545088:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315366.628246:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307544960:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:02020000:2.0:1761315366.631697:0:2262:0:(client.c:1361:ptlrpc_check_status()) 11-0: lustre-OST0000-osc-ffff951bc4d8c800: operation ost_connect to node 192.168.203.151@tcp failed: rc = -19 00000100:00080000:2.0:1761315366.631714:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761315366.631733:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315366.631738:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307545152:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761315366.631845:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307545024:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315366.637351:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307545088:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315366.883382:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315366.945978:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315367.904912:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315367.969134:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315368.930698:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315368.993667:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315369.957790:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315370.016943:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315370.976775:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315371.040910:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761315371.746279:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315371.746287:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761315371.746290:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761315371.746300:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3614 00000100:00080000:1.0:1761315371.746304:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761315371.746309:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761315371.746319:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761315371.746334:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a1719d14 x1846869307545408/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761315371.746344:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307545216:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761315371.746362:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315371.746365:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761315371.746407:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307545280:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315371.746423:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307545344:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315371.746663:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307545408:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00080000:2.0:1761315371.754797:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761315371.754809:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315371.754818:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307545408:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761315371.755349:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307545280:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315371.755466:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307545216:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315371.755476:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307545344:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315372.000776:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315372.064892:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315373.025164:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315373.091994:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315374.050978:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315374.112889:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315375.073331:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315375.137875:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315376.096782:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315376.162123:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761315376.865537:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315376.865544:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761315376.865547:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761315376.865557:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3619 00000100:00080000:3.0:1761315376.865560:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00100000:2.0:1761315376.865564:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004e00d332 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307545472:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761315376.865566:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761315376.865575:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761315376.865588:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000004505177d x1846869307545664/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761315376.865694:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307545664:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00080000:3.0:1761315376.865759:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315376.865762:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761315376.865790:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cdcc674d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307545536:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761315376.867481:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000038534f0b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307545600:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761315376.872502:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004e00d332 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307545472:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761315376.874448:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000038534f0b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307545600:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761315376.874589:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761315376.874594:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761315376.874599:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307545664:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761315376.874757:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cdcc674d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307545536:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761315377.121016:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315377.186241:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315378.146174:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315378.209491:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315379.168790:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315379.232915:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315380.192787:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315380.257860:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315381.217194:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315381.280864:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315381.986085:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cdcc674d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307545728:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315381.986279:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761315381.986285:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000626f98db pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307545792:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761315381.986795:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315381.986801:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761315381.986804:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761315381.986809:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3624 00000100:00080000:3.0:1761315381.986812:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761315381.986818:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761315381.986827:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761315381.986842:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000038534f0b x1846869307545920/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761315381.986862:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315381.986865:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761315381.986879:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761315381.986889:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307545856:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315381.987142:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000038534f0b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307545920:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315381.994487:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000626f98db pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307545792:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315381.994505:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cdcc674d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307545728:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315382.002737:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307545856:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761315382.002834:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761315382.002838:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761315382.002843:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000038534f0b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307545920:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315382.241775:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315382.305948:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315383.266341:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315383.329112:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315384.288957:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315384.353038:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315385.312982:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315385.376918:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315386.338242:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315386.401182:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761315387.104911:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315387.104918:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761315387.104921:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761315387.104931:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3630 00000100:00080000:3.0:1761315387.104947:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761315387.104953:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761315387.104962:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761315387.104979:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006c8fcfe5 x1846869307546176/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761315387.105009:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315387.105013:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761315387.105205:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307546176:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761315387.108419:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307545984:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761315387.108539:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307546112:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761315387.108584:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761315387.108591:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004e00d332 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307546048:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761315387.114751:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761315387.114762:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761315387.114772:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307546176:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761315387.115209:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307545984:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761315387.115226:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307546112:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761315387.115246:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004e00d332 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307546048:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761315387.360962:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315387.426288:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315388.386977:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315388.449934:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315389.409773:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315389.472916:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315390.433985:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315390.499017:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315391.456906:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315391.525446:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761315392.224962:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315392.224973:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761315392.224977:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761315392.224990:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3635 00000100:00080000:0.0:1761315392.224993:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761315392.225000:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761315392.225013:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761315392.225033:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008e59d025 x1846869307546432/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761315392.225080:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315392.225085:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761315392.225119:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307546304:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761315392.225131:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307546432:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761315392.225237:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1761315392.225244:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307546240:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761315392.225694:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307546368:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761315392.236085:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761315392.236097:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315392.236110:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307546432:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761315392.236208:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307546240:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761315392.236244:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307546368:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761315392.236254:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307546304:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761315392.481529:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315392.544909:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315393.504808:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315393.568919:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315394.529129:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315394.595792:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315395.555251:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315395.618921:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315396.576802:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315396.640938:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761315397.345538:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315397.345548:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761315397.345552:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761315397.345566:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3640 00000100:00080000:3.0:1761315397.345590:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761315397.345598:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761315397.345612:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761315397.345632:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000602b09d0 x1846869307546688/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761315397.345698:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315397.345702:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761315397.345737:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307546560:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315397.345849:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1761315397.345855:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307546496:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315397.345881:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307546624:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315397.345991:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307546688:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761315397.349158:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307546496:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315397.349190:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307546624:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315397.349202:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307546560:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761315397.352672:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761315397.352686:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761315397.352698:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307546688:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761315397.601931:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315397.665283:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315398.624797:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315398.689944:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315399.648826:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315399.714624:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315400.672823:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315400.738217:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315401.697547:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315401.761801:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761315402.465369:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315402.465380:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761315402.465384:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761315402.465398:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3645 00000100:00080000:2.0:1761315402.465402:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761315402.465428:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761315402.465441:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761315402.465471:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008a641e2e x1846869307546944/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761315402.465500:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315402.465503:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761315402.465537:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ffb03e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307546816:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761315402.465713:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1761315402.465720:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e1be9a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307546752:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761315402.465748:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e662bceb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307546880:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761315402.467091:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008a641e2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307546944:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761315402.476602:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ffb03e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307546816:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761315402.476655:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e1be9a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307546752:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761315402.476666:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e662bceb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307546880:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761315402.482877:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761315402.482892:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761315402.482902:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008a641e2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307546944:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761315402.721174:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315402.784919:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315403.745166:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315403.809371:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315404.769158:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315404.835728:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315405.796768:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315405.857052:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315406.816797:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315406.882159:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761315407.584966:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315407.584976:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761315407.584979:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761315407.584992:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3650 00000100:00080000:3.0:1761315407.584996:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761315407.585003:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761315407.585026:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761315407.585045:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ddd3c50a x1846869307547200/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761315407.585070:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315407.585073:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761315407.585108:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307547072:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761315407.585239:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1761315407.585246:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307547008:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761315407.585273:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307547136:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761315407.585685:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307547200:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761315407.600042:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307547072:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761315407.600074:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307547008:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761315407.600103:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307547136:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761315407.601215:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761315407.601222:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761315407.601230:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307547200:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761315407.841182:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315407.905179:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315408.864757:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315408.928920:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315409.891979:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315409.952872:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315410.913119:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315410.976836:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315411.937724:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315412.001035:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761315412.704983:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315412.704994:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761315412.704998:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761315412.705010:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3655 00000100:00080000:3.0:1761315412.705013:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761315412.705021:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761315412.705035:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761315412.705053:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f133bb2e x1846869307547456/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761315412.705086:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315412.705090:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761315412.705527:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307547264:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315412.705633:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307547392:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315412.706140:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761315412.706155:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307547328:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315412.709492:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f133bb2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307547456:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315412.718774:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307547392:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315412.718807:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307547328:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761315412.719670:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761315412.719676:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761315412.719682:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f133bb2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307547456:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761315412.719793:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307547264:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315412.961061:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315413.024855:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315413.984756:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315414.048851:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315415.010455:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315415.072888:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315416.033242:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315416.096910:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315417.058255:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315417.120864:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315417.828082:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e662bceb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307547520:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761315417.830682:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315417.830691:0:18179:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761315417.830696:0:18179:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761315417.830708:0:18179:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3660 00000100:00080000:2.0:1761315417.830712:0:18179:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00100000:3.0:1761315417.830876:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e1be9a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307547584:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761315417.830967:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761315417.830974:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ffb03e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307547648:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761315417.831455:0:18179:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761315417.831470:0:18179:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761315417.831491:0:18179:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ee08055d x1846869307547712/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761315417.831555:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ee08055d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307547712:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00080000:2.0:1761315417.832590:0:18179:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315417.832597:0:18179:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761315417.834107:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e662bceb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307547520:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761315417.836416:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e1be9a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307547584:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761315417.836446:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ffb03e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307547648:192.168.203.151@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761315417.836857:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761315417.836863:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761315417.836869:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ee08055d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307547712:192.168.203.151@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761315418.082980:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315418.145603:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315419.106993:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315419.168874:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315420.131272:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315420.192863:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315421.153060:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315421.219472:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315422.177412:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315422.243272:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761315422.945564:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315422.945572:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761315422.945576:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761315422.945587:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3665 00000100:00080000:1.0:1761315422.945590:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00100000:0.0:1761315422.945592:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307547776:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761315422.945599:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761315422.945608:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761315422.945626:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a1719d14 x1846869307547968/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761315422.945689:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315422.945692:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761315422.945712:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307547904:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315422.945716:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307547840:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315422.945826:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307547968:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761315422.952958:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307547840:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315422.952996:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307547776:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315422.953014:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307547904:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761315422.961172:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761315422.961183:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315422.961194:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307547968:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761315423.201052:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315423.265367:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315424.224777:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315424.292089:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315425.248758:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315425.314663:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315426.272972:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315426.339164:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315427.298559:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315427.360866:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761315428.064860:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315428.064866:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761315428.064869:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761315428.064880:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3671 00000100:00080000:1.0:1761315428.064883:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761315428.064888:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761315428.064897:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761315428.064910:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000eb8b1f19 x1846869307548224/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761315428.064930:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315428.064933:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761315428.064952:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307548096:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315428.065045:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1761315428.065050:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307548032:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315428.065069:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307548160:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315428.067199:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307548224:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761315428.079198:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307548032:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315428.079217:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307548160:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315428.079237:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307548096:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761315428.080262:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761315428.080269:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315428.080275:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307548224:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761315428.320794:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315428.386277:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315429.344770:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315429.408836:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315430.374815:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315430.434001:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315431.394986:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315431.456960:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315432.416772:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315432.481021:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761315433.186087:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315433.186094:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761315433.186097:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761315433.186107:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3676 00000100:00080000:3.0:1761315433.186110:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761315433.186127:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761315433.186136:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761315433.186152:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000cc6dd8ca x1846869307548480/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761315433.186176:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315433.186179:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761315433.186223:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307548352:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315433.186225:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cc6dd8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307548480:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761315433.187211:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1761315433.187225:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307548288:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315433.187287:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307548416:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761315433.193668:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761315433.193678:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761315433.193688:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc6dd8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307548480:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761315433.194394:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307548288:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315433.194410:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307548352:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315433.195555:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307548416:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315433.440795:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315433.504867:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315434.464769:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315434.529175:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315435.488804:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315435.555347:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315436.512911:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315436.576835:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315437.536801:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315437.600892:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761315438.304831:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315438.304839:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761315438.304842:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761315438.304853:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3681 00000100:00080000:3.0:1761315438.304856:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761315438.304861:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761315438.304872:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761315438.304886:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006e93398e x1846869307548736/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761315438.304910:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315438.304913:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761315438.305004:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307548544:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315438.305089:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307548672:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315438.305119:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761315438.305123:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307548608:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315438.305261:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307548736:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00080000:3.0:1761315438.310310:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761315438.310344:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761315438.310357:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307548736:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315438.310738:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307548608:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315438.310752:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307548672:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315438.311788:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307548544:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315438.560775:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315438.624859:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315439.587814:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315439.648846:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315440.608784:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315440.672869:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315441.632784:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315441.697018:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315442.656833:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315442.722037:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761315443.424848:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315443.424855:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761315443.424858:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761315443.424867:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3686 00000100:00080000:3.0:1761315443.424870:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761315443.424875:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761315443.424884:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761315443.424898:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000101f852a x1846869307548992/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761315443.424924:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315443.424928:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761315443.424999:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307548800:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315443.425088:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f597cda6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307548928:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315443.425120:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761315443.425125:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f364ec74 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307548864:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315443.425577:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000101f852a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307548992:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00080000:1.0:1761315443.438049:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:2.0:1761315443.438057:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f364ec74 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307548864:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761315443.438059:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761315443.438066:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000101f852a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307548992:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315443.438073:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307548800:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315443.438085:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f597cda6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307548928:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315443.680765:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315443.747203:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315444.704750:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315444.768974:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315445.728788:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315445.793679:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315446.753222:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315446.818098:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315447.776865:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315447.840810:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761315448.544863:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315448.544871:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761315448.544874:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761315448.544884:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3691 00000100:00080000:3.0:1761315448.544887:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761315448.544893:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761315448.544902:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761315448.544916:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d32bed84 x1846869307549248/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761315448.544957:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315448.544960:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761315448.545013:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d32bed84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307549248:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315448.545064:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000009911ad3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307549120:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315448.545442:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1761315448.545449:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307549056:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315448.545475:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007673f690 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307549184:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761315448.549339:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761315448.549353:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761315448.549374:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d32bed84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307549248:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315448.549374:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307549056:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315448.549395:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007673f690 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307549184:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315448.549406:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000009911ad3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307549120:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315448.801825:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315448.865492:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315449.825030:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315449.889388:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315450.849935:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315450.912839:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315451.872799:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315451.937286:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315452.896772:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315452.960946:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315453.665267:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b3aaddc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307549312:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761315453.670155:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315453.670164:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761315453.670167:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761315453.670180:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3696 00000100:00080000:2.0:1761315453.670184:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761315453.670191:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761315453.670203:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761315453.670219:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ba5c8e77 x1846869307549504/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761315453.670300:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ba5c8e77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307549504:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761315453.671166:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001426be00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307549440:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761315453.671269:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761315453.671277:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000011b5c99f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307549376:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761315453.671517:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315453.671522:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761315453.672135:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b3aaddc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307549312:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761315453.675063:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761315453.675079:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761315453.675089:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ba5c8e77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307549504:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761315453.675311:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001426be00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307549440:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761315453.675549:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000011b5c99f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307549376:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315453.921468:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315453.985173:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315454.945599:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315455.009711:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315455.970177:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315456.032925:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315456.992836:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315457.056845:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315458.016801:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315458.081031:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761315458.789822:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315458.789829:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761315458.789833:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761315458.789843:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3701 00000100:00100000:0.0:1761315458.789844:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307549568:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761315458.789848:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761315458.789857:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761315458.789866:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761315458.789882:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000eb8b1f19 x1846869307549760/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761315458.789902:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315458.789905:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761315458.789936:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307549696:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315458.789969:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761315458.789975:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307549632:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315458.790036:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307549760:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761315458.798622:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307549632:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315458.798686:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307549568:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315458.798702:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307549696:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761315458.799326:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761315458.799337:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315458.799347:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307549760:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315459.041874:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315459.104947:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315460.064798:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315460.129687:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315461.088764:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315461.153001:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315462.113066:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315462.177314:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315463.137777:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315463.203146:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761315463.904885:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315463.904894:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761315463.904899:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761315463.904912:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3706 00000100:00080000:3.0:1761315463.904917:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761315463.904924:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000100:00100000:2.0:1761315463.904928:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000041c8b6cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307549824:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315463.905090:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000009911ad3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307549952:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315463.905129:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761315463.905135:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e58cc722 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307549888:192.168.203.151@tcp:400:kworker/u8:1.0 00000008:00100000:3.0:1761315463.905371:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761315463.905418:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007673f690 x1846869307550016/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761315463.905476:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315463.905480:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761315463.905742:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007673f690 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307550016:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315463.909024:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000041c8b6cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307549824:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315463.909064:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000009911ad3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307549952:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315463.909098:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e58cc722 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307549888:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761315463.914850:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761315463.914866:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315463.914878:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007673f690 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307550016:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315464.160807:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315464.224894:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315465.185560:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315465.249079:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315466.209334:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315466.272955:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315467.233838:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315467.298967:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315468.256749:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315468.321342:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761315469.024906:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315469.024914:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761315469.024917:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761315469.024929:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3711 00000100:00080000:1.0:1761315469.024932:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761315469.024938:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761315469.024948:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761315469.024965:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000540c3ce6 x1846869307550272/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761315469.025001:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315469.025005:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761315469.025033:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307550144:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315469.025127:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1761315469.025133:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307550080:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315469.025155:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307550208:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761315469.025451:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307550272:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761315469.029119:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307550080:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315469.029143:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307550144:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315469.029684:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307550208:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761315469.030306:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761315469.030311:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315469.030316:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307550272:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761315469.280757:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315469.344863:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315470.304735:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315470.368832:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315471.328759:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315471.393720:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315472.352759:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315472.417065:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315473.376799:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315473.445588:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761315474.146451:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315474.146458:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761315474.146461:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761315474.146472:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3717 00000100:00080000:0.0:1761315474.146475:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761315474.146480:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761315474.146489:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761315474.146504:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000005a420586 x1846869307550528/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761315474.146529:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315474.146532:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761315474.146568:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307550400:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761315474.146571:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307550528:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761315474.146677:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1761315474.146684:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307550336:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315474.146708:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307550464:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315474.152068:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307550400:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761315474.159292:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761315474.159308:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315474.159320:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307550528:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761315474.159987:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307550336:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315474.160691:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307550464:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761315474.401267:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315474.465384:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315475.426277:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315475.491050:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315476.448754:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315476.512864:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315477.476174:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315477.536861:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315478.496777:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315478.560909:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761315479.264829:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315479.264839:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761315479.264843:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761315479.264855:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3722 00000100:00080000:0.0:1761315479.264859:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761315479.264865:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761315479.264875:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761315479.264892:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008e59d025 x1846869307550784/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761315479.264913:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315479.264916:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761315479.265020:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307550656:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315479.265128:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1761315479.265134:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307550592:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315479.265157:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307550720:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761315479.265257:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307550784:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761315479.277034:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307550656:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315479.278463:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307550592:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315479.278482:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307550720:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761315479.280192:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761315479.280202:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315479.280211:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307550784:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761315479.520739:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315479.587486:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315480.548060:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315480.612151:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315481.586759:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315481.635147:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315482.592770:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315482.656863:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315483.616766:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315483.682135:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761315484.385308:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315484.385318:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761315484.385322:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761315484.385335:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3727 00000100:00080000:2.0:1761315484.385339:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761315484.385345:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761315484.385358:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761315484.385377:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006e3466f5 x1846869307551040/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761315484.385445:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e3466f5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307551040:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761315484.385627:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000011b5c99f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307550848:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315484.385680:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000015f165e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307550976:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315484.385704:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761315484.385708:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001426be00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307550912:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761315484.388250:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315484.388264:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00080000:2.0:1761315484.392597:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761315484.392612:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315484.392625:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e3466f5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307551040:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761315484.392836:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001426be00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307550912:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315484.392849:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000011b5c99f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307550848:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315484.392860:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000015f165e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307550976:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315484.640769:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315484.704964:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315485.664794:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315485.731293:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315486.689217:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315486.753525:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315487.713375:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315487.778428:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315488.743370:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315488.800880:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761315489.504927:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315489.504936:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761315489.504940:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761315489.504954:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3732 00000100:00080000:1.0:1761315489.504958:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761315489.504965:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761315489.504977:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761315489.504993:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a1719d14 x1846869307551296/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761315489.505052:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315489.505056:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761315489.505087:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307551168:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315489.505193:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1761315489.505200:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307551104:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315489.505226:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307551232:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761315489.505662:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307551296:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761315489.511742:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307551104:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761315489.514348:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761315489.514362:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315489.514372:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307551296:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761315489.514454:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307551168:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315489.514472:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307551232:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315489.760804:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315489.832619:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315490.784974:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315490.852318:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315491.820335:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315491.873182:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315492.833115:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315492.897759:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315493.856816:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315493.922284:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761315494.625332:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315494.625341:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761315494.625344:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761315494.625356:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3737 00000100:00080000:2.0:1761315494.625360:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761315494.625366:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761315494.625378:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761315494.625396:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ae09c97d x1846869307551552/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761315494.625424:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315494.625428:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761315494.625718:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e3466f5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307551360:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315494.625849:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004592c0a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307551488:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315494.625882:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761315494.625906:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039978a4e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307551424:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315494.625993:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ae09c97d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307551552:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761315494.631741:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e3466f5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307551360:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315494.636254:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039978a4e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307551424:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315494.636276:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004592c0a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307551488:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761315494.637340:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761315494.637351:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315494.637360:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ae09c97d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307551552:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761315494.880995:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315494.947311:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315495.904963:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315495.970480:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315496.928774:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315496.995302:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315497.952781:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315498.016877:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315498.977099:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315499.041042:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315499.744955:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307551616:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761315499.744999:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315499.745004:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761315499.745008:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1761315499.745074:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307551744:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761315499.745083:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3742 00000100:00080000:0.0:1761315499.745087:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00100000:1.0:1761315499.745101:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761315499.745106:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307551680:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761315499.745213:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761315499.745361:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761315499.745396:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000546b1bfe x1846869307551808/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761315499.745481:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307551808:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00080000:0.0:1761315499.746395:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315499.746402:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761315499.751835:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307551680:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315499.751879:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307551744:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315499.751890:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307551616:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761315499.752255:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761315499.752262:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315499.752267:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307551808:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761315500.000950:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315500.065477:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315501.024785:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315501.091081:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315502.049101:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315502.113497:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315503.072779:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315503.137017:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315504.097926:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315504.161717:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761315504.866116:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315504.866123:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761315504.866137:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761315504.866147:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3747 00000100:00080000:1.0:1761315504.866150:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761315504.866156:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761315504.866165:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761315504.866180:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a1719d14 x1846869307552064/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761315504.866205:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315504.866208:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761315504.866265:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307551872:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315504.866345:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307552000:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315504.866374:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761315504.866379:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307551936:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761315504.866687:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307552064:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00080000:2.0:1761315504.870818:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761315504.870829:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315504.870910:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307552064:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761315504.871593:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307551936:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315504.871605:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307551872:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315504.871614:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307552000:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315505.120783:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315505.187238:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315506.144876:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315506.209664:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315507.168781:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315507.232895:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315508.193124:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315508.257101:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315509.216923:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315509.281146:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761315509.984988:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315509.984995:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761315509.984998:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761315509.985008:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3752 00000100:00080000:2.0:1761315509.985011:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761315509.985017:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761315509.985027:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761315509.985042:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000039eebc2 x1846869307552320/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761315509.985067:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315509.985071:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761315509.986346:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000039eebc2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307552320:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761315509.986346:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ae09c97d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307552128:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315509.986746:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eca34e1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307552192:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315509.987801:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e9376149 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307552256:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315509.991220:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ae09c97d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307552128:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315510.000303:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e9376149 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307552256:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315510.001779:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eca34e1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307552192:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761315510.010809:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761315510.010817:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761315510.010825:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000039eebc2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307552320:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315510.240807:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315510.305063:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315511.264823:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315511.328920:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315512.289748:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315512.353053:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315513.312892:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315513.377377:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315514.337002:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315514.401236:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315515.105610:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307552384:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315515.107367:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307552448:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315515.107873:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761315515.107879:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307552512:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761315515.108411:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315515.108419:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761315515.108424:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761315515.108431:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3758 00000100:00080000:1.0:1761315515.108435:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761315515.108441:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761315515.108454:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761315515.108477:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000eb8b1f19 x1846869307552576/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761315515.108502:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315515.108506:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761315515.109116:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307552576:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761315515.111126:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307552512:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315515.111150:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307552448:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315515.111706:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307552384:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761315515.113248:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761315515.113260:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315515.113268:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307552576:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761315515.362975:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315515.428887:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315516.386060:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315516.449061:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315517.411790:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315517.472932:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315518.432826:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315518.498696:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315519.458797:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315519.521435:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761315520.224874:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315520.224887:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761315520.224891:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761315520.224901:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3763 00000100:00080000:0.0:1761315520.224904:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761315520.224911:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761315520.224923:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761315520.224940:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000005b900d12 x1846869307552832/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761315520.224962:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315520.224966:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761315520.224989:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307552640:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315520.225007:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307552704:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315520.225064:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307552832:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761315520.225083:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307552768:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315520.232983:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307552768:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:02020000:2.0:1761315520.233124:0:2262:0:(client.c:1361:ptlrpc_check_status()) 11-0: lustre-OST0000-osc-ffff951bc4d8c800: operation ost_connect to node 192.168.203.151@tcp failed: rc = -19 00000100:00080000:2.0:1761315520.233134:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761315520.233138:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315520.233142:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307552832:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761315520.233346:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307552704:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315520.234203:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307552640:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315520.480805:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315520.544933:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315521.504823:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315521.569780:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315522.528870:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315522.593677:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315523.552798:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315523.616907:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315524.576827:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315524.640887:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761315525.345010:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315525.345020:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761315525.345024:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761315525.345037:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3768 00000100:00080000:3.0:1761315525.345041:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761315525.345067:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761315525.345080:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761315525.345101:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006e93398e x1846869307553088/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761315525.345130:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315525.345133:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761315525.345162:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d32bed84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307552960:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761315525.345283:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1761315525.345290:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307552896:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761315525.345316:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f597cda6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307553024:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315525.346017:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307553088:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761315525.349928:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f597cda6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307553024:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761315525.349952:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d32bed84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307552960:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761315525.350081:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761315525.350087:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761315525.350093:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307553088:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761315525.351289:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307552896:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761315525.600801:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315525.665177:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315526.624829:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315526.688961:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315527.648838:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315527.713218:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315528.673141:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315528.738282:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315529.697021:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315529.762070:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761315530.464890:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315530.464896:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761315530.464899:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761315530.464909:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3773 00000100:00080000:0.0:1761315530.464913:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761315530.464918:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761315530.464927:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761315530.464941:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000028de4b9d x1846869307553344/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761315530.464965:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315530.464968:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761315530.465044:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307553152:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315530.465131:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307553280:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315530.465161:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761315530.465166:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307553216:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315530.465264:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307553344:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761315530.470720:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307553216:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315530.470809:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307553152:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315530.470845:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307553280:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761315530.470968:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761315530.470972:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315530.470977:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307553344:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315530.721108:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315530.790962:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315531.744974:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315531.809065:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315532.768766:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315532.833568:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315533.792967:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315533.859217:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315534.818860:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315534.895057:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761315535.584892:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315535.584899:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761315535.584903:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1761315535.584915:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307553408:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761315535.584915:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3778 00000100:00080000:1.0:1761315535.584921:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761315535.584944:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761315535.584955:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761315535.584972:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000540c3ce6 x1846869307553600/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761315535.584997:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315535.585001:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761315535.585052:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307553472:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315535.585066:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307553536:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315535.585423:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307553600:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761315535.590372:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307553408:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315535.592431:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307553536:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315535.592997:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307553472:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761315535.595379:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761315535.595394:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315535.595405:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307553600:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315535.843676:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315535.905152:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315536.864816:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315536.928911:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315537.889040:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315537.953608:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315538.912765:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315538.976907:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315539.936764:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315540.000934:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761315540.704799:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315540.704807:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761315540.704810:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761315540.704821:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3783 00000100:00080000:1.0:1761315540.704825:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761315540.704831:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761315540.704841:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761315540.704857:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ed2f9bca x1846869307553856/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761315540.704885:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315540.704889:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761315540.704912:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307553728:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315540.705014:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1761315540.705021:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307553664:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315540.705046:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307553792:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761315540.706195:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307553856:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761315540.712289:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307553664:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315540.712310:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307553792:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315540.712320:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307553728:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761315540.716343:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761315540.716354:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315540.716364:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307553856:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761315540.960738:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315541.025363:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315541.986768:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315542.048872:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315543.008853:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315543.072897:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315544.036018:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315544.097120:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315545.056784:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315545.120905:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761315545.827130:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315545.827137:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761315545.827140:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761315545.827152:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3788 00000100:00080000:0.0:1761315545.827155:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761315545.827161:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761315545.827170:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761315545.827198:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000034a6cd79 x1846869307554112/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761315545.827224:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315545.827227:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761315545.827253:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307553984:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315545.827277:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307554112:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761315545.827403:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307553920:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315545.827910:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307554048:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315545.833389:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307553984:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315545.833475:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307553920:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761315545.833483:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:0.0:1761315545.833487:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307554048:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761315545.833488:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315545.833492:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307554112:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315546.080954:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315546.145876:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315547.105201:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315547.171050:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315548.129546:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315548.192890:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315549.152828:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315549.216917:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315550.176808:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315550.241947:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761315550.947237:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315550.947247:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761315550.947251:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761315550.947266:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3793 00000100:00080000:0.0:1761315550.947270:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761315550.947277:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761315550.947289:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761315550.947309:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000034a6cd79 x1846869307554368/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761315550.947344:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315550.947348:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761315550.947380:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307554240:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315550.947461:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307554368:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761315550.947517:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1761315550.947547:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307554176:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315550.947576:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307554304:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315550.953594:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307554304:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315550.953618:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307554240:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761315550.957708:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:0.0:1761315550.957709:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307554176:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761315550.957721:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315550.957728:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307554368:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315551.200817:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315551.266629:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315552.224819:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315552.289308:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315553.250163:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315553.316991:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315554.272785:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315554.336879:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315555.296823:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315555.361663:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315556.064964:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307554432:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315556.065684:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761315556.065696:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307554496:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761315556.066719:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315556.066727:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761315556.066730:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761315556.066738:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3799 00000100:00080000:0.0:1761315556.066742:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761315556.066748:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761315556.066762:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761315556.066781:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007dcceb4a x1846869307554624/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761315556.066806:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315556.066810:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761315556.066827:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761315556.066842:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307554560:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315556.066956:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307554624:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761315556.069076:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307554560:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761315556.069242:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761315556.069248:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761315556.069253:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307554624:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761315556.069688:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307554496:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315556.069701:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307554432:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315556.320816:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315556.387324:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315557.344725:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315557.409842:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315558.368822:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315558.434876:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315559.392724:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315559.456805:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315560.416776:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315560.481296:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761315561.187729:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315561.187736:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761315561.187740:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761315561.187751:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3804 00000100:00080000:1.0:1761315561.187770:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761315561.187776:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761315561.187786:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761315561.187803:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000003871442b x1846869307554880/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761315561.187827:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315561.187831:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761315561.187854:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307554752:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315561.187939:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1761315561.187946:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307554688:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315561.187968:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307554816:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761315561.188055:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307554880:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00080000:3.0:1761315561.190903:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761315561.190921:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761315561.190935:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307554880:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761315561.190964:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307554688:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315561.190991:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307554816:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315561.191002:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307554752:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761315561.441786:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315561.504869:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315562.466128:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315562.531183:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315563.488890:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315563.552924:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315564.512821:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315564.576920:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315565.536792:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315565.600862:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761315566.304972:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315566.304981:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761315566.304984:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761315566.304998:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3809 00000100:00080000:0.0:1761315566.305005:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761315566.305012:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761315566.305023:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761315566.305043:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000005b900d12 x1846869307555136/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761315566.305066:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315566.305069:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761315566.305121:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307555008:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315566.305226:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1761315566.305232:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307554944:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315566.305266:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307555072:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315566.305576:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307555136:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761315566.311036:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307554944:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315566.311732:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307555072:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315566.311769:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307555008:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761315566.313516:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761315566.313532:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761315566.313543:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307555136:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315566.561105:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315566.627399:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315567.587633:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315567.652713:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315568.609396:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315568.677243:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315569.632821:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315569.700936:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315570.659215:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315570.721574:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761315571.431331:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315571.431341:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761315571.431345:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761315571.431358:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3814 00000100:00080000:1.0:1761315571.431382:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761315571.431390:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761315571.431402:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761315571.431421:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000eb8b1f19 x1846869307555392/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761315571.431446:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315571.431449:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761315571.431483:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307555264:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315571.431592:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1761315571.431598:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307555200:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315571.431624:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307555328:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761315571.433394:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307555392:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761315571.436589:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307555264:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315571.437663:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307555328:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315571.437971:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307555200:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761315571.440458:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761315571.440470:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761315571.440481:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307555392:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761315571.680991:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315571.745536:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315572.704832:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315572.770185:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315573.728818:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315573.792935:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315574.752921:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315574.819886:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315575.777351:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315575.841471:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761315576.545159:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315576.545168:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761315576.545172:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761315576.545185:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3819 00000100:00080000:0.0:1761315576.545189:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761315576.545196:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761315576.545210:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761315576.545230:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000084f3d917 x1846869307555648/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761315576.545268:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315576.545271:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761315576.545305:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307555520:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315576.545439:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1761315576.545445:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307555456:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315576.545470:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307555584:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315576.546192:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307555648:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761315576.548443:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307555456:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315576.548467:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307555584:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315576.548478:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307555520:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761315576.549761:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761315576.549770:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761315576.549801:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307555648:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315576.800947:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315576.874414:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315577.825770:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315577.888910:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315578.849056:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315578.916675:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315579.873058:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315579.938038:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315580.897430:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315580.962517:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315581.664960:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eca34e1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307555712:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761315581.665021:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315581.665027:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761315581.665042:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761315581.665048:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3824 00000100:00080000:2.0:1761315581.665051:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761315581.665057:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761315581.665070:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761315581.665089:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f220504f x1846869307555904/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1761315581.665098:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aea3302e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307555840:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761315581.665122:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315581.665125:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761315581.665134:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761315581.665139:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b2298797 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307555776:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315581.665295:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f220504f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307555904:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761315581.671117:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eca34e1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307555712:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761315581.673387:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761315581.673402:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761315581.673417:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f220504f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307555904:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761315581.673872:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b2298797 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307555776:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315581.673888:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aea3302e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307555840:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315581.920774:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315581.986353:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315582.944940:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315583.009036:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315583.969307:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315584.032856:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315584.992852:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315585.057329:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315586.016777:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315586.082973:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761315586.785069:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315586.785078:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761315586.785081:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761315586.785095:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3829 00000100:00080000:0.0:1761315586.785099:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761315586.785105:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761315586.785117:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761315586.785136:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000057b18867 x1846869307556160/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761315586.785210:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315586.785214:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761315586.785249:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307556032:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315586.785257:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307556160:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761315586.785327:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307555968:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315586.785688:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307556096:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761315586.792220:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761315586.792235:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761315586.792248:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307556160:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761315586.800986:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307556032:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315586.805265:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307555968:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315586.805294:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307556096:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315587.040818:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315587.105120:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315588.066288:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315588.130080:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315589.088768:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315589.155102:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315590.112845:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315590.179277:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315590.884021:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307556224:192.168.203.151@tcp:17:kworker/1:0.0 00000100:00100000:1.0:1761315590.884098:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761315590.884103:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307556288:192.168.203.151@tcp:17:kworker/1:0.0 00000100:00100000:1.0:1761315590.888771:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307556224:192.168.203.151@tcp:17:kworker/1:0.0 00000100:00100000:1.0:1761315590.888803:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307556288:192.168.203.151@tcp:17:kworker/1:0.0 00000100:00100000:0.0:1761315591.136782:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315591.202156:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315591.904977:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307556352:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315591.905571:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761315591.905581:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307556416:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761315591.905920:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315591.905925:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761315591.905929:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761315591.905935:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3834 00000100:00080000:1.0:1761315591.905939:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761315591.905944:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761315591.905974:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761315591.905993:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000eb8b1f19 x1846869307556544/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761315591.906019:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315591.906022:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761315591.906036:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761315591.906046:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307556480:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761315591.906180:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307556544:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761315591.909768:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307556352:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315591.909809:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307556416:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761315591.910083:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761315591.910089:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761315591.910094:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307556544:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761315591.910168:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307556480:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315592.160769:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315592.224861:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315593.184761:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315593.248914:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315594.208819:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315594.273624:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315595.232765:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315595.297351:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315596.257057:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315596.321374:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761315597.024917:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315597.024924:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761315597.024928:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761315597.024952:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3839 00000100:00080000:2.0:1761315597.024955:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761315597.024962:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761315597.024972:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761315597.024988:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b4eac89a x1846869307556800/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761315597.025016:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315597.025019:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761315597.025144:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002292ab57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307556608:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315597.025247:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000007136630 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307556736:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315597.025293:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761315597.025300:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307556672:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315597.025671:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b4eac89a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307556800:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00080000:1.0:1761315597.032685:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761315597.032694:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761315597.032703:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b4eac89a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307556800:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761315597.032784:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307556672:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315597.032794:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002292ab57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307556608:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315597.032803:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000007136630 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307556736:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315597.280768:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315597.354511:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315598.304842:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315598.368972:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315599.329318:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315599.395312:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315600.352753:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315600.417793:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315601.376779:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315601.443498:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761315602.145245:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315602.145253:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761315602.145257:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761315602.145270:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3845 00000100:00080000:3.0:1761315602.145275:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761315602.145281:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761315602.145292:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761315602.145307:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f364ec74 x1846869307557056/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761315602.145328:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315602.145331:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761315602.145361:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d32bed84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307556928:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315602.145433:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f364ec74 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307557056:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761315602.146262:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307556864:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761315602.149234:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f597cda6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307556992:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761315602.166580:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761315602.166592:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761315602.166601:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f364ec74 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307557056:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761315602.166766:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d32bed84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307556928:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761315602.166873:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307556864:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761315602.166888:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f597cda6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307556992:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315602.400831:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315602.464967:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315603.438083:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315603.488783:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315604.448782:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315604.512917:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315605.472773:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315605.538250:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315606.498053:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315606.562175:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315607.265835:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307557120:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315607.267559:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307557184:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315607.268005:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761315607.268011:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307557248:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761315607.268166:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315607.268171:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761315607.268174:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761315607.268179:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3850 00000100:00080000:1.0:1761315607.268181:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761315607.268186:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761315607.268197:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761315607.268213:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000003871442b x1846869307557312/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761315607.268237:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315607.268240:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761315607.268358:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307557312:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761315607.270732:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307557120:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315607.273568:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307557248:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315607.273588:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307557184:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761315607.273746:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761315607.273763:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761315607.273768:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307557312:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761315607.520868:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315607.585534:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315608.544757:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315608.609842:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315609.568925:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315609.634310:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315610.592877:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315610.657056:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315611.616795:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315611.682464:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761315612.386872:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315612.386881:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761315612.386901:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761315612.386915:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3855 00000100:00080000:0.0:1761315612.386919:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761315612.386925:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761315612.386936:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761315612.386958:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000005a420586 x1846869307557568/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761315612.386993:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315612.386997:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761315612.387080:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307557376:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315612.387187:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307557504:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315612.387238:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761315612.387245:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307557440:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315612.388124:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307557568:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761315612.391990:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307557376:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315612.408752:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307557504:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315612.408796:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307557440:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761315612.409143:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761315612.409150:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761315612.409155:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307557568:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761315612.640804:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315612.705560:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315613.664883:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315613.729057:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315614.688843:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315614.752849:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315615.712805:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315615.776999:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315616.736772:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315616.801819:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761315617.508234:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315617.508244:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761315617.508248:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761315617.508260:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3860 00000100:00080000:1.0:1761315617.508265:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761315617.508272:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761315617.508285:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761315617.508304:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000003871442b x1846869307557824/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761315617.508335:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315617.508338:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761315617.508374:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307557696:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315617.508497:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1761315617.508505:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307557632:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315617.508531:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307557760:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761315617.508869:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307557824:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761315617.516475:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307557696:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315617.516531:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307557632:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761315617.517349:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761315617.517355:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761315617.517361:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307557824:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761315617.517432:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307557760:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315617.761698:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315617.827367:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315618.791123:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315618.850502:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315619.812082:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315619.872896:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315620.833243:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315620.897341:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315621.856802:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315621.920910:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761315622.625731:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315622.625739:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761315622.625743:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761315622.625755:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3865 00000100:00080000:1.0:1761315622.625759:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761315622.625766:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761315622.625776:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761315622.625793:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000eb8b1f19 x1846869307558080/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761315622.625814:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315622.625817:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761315622.627024:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307558080:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761315622.627248:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307557952:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315622.628215:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1761315622.628227:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307557888:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315622.628284:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307558016:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315622.634198:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307557888:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315622.636177:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307557952:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761315622.636444:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761315622.636450:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761315622.636455:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307558080:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761315622.643696:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307558016:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315622.880797:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315622.944880:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315623.910192:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315623.969674:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315624.934317:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315624.992875:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315625.953725:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315626.016943:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315626.976793:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315627.040907:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761315627.750538:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315627.750556:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761315627.750560:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761315627.750574:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3870 00000100:00080000:2.0:1761315627.750578:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761315627.750586:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761315627.750599:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761315627.750619:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b4eac89a x1846869307558336/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761315627.750724:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315627.750728:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761315627.750764:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002292ab57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307558208:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315627.750883:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1761315627.750889:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000007136630 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307558144:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315627.750910:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307558272:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315627.752057:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b4eac89a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307558336:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00080000:1.0:1761315627.760436:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761315627.760451:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761315627.760465:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b4eac89a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307558336:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315627.761205:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000007136630 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307558144:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315627.762013:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307558272:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315627.762094:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002292ab57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307558208:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315628.001243:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315628.067416:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315629.027157:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315629.088926:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315630.048770:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315630.113949:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315631.072818:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315631.136928:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315632.098566:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315632.160962:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761315632.866705:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315632.866713:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761315632.866716:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761315632.866727:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3875 00000100:00080000:2.0:1761315632.866730:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761315632.866736:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761315632.866747:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761315632.866761:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000aea3302e x1846869307558592/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761315632.866857:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aea3302e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307558592:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761315632.867689:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002292ab57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307558400:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761315632.867934:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315632.867941:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761315632.868019:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307558464:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761315632.868942:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000007136630 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307558528:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761315632.872083:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761315632.872106:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761315632.872118:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aea3302e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307558592:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761315632.876229:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002292ab57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307558400:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761315632.879933:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307558464:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761315632.880127:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000007136630 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307558528:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315633.120796:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315633.185203:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315634.144854:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315634.209930:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315635.168809:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315635.233404:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315636.192825:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315636.256942:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315637.216833:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315637.284594:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315637.985098:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d32bed84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307558656:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315637.985873:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761315637.985885:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000101f852a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307558720:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315637.992777:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d32bed84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307558656:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761315637.993004:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315637.993029:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761315637.993033:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761315637.993038:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3880 00000100:00080000:3.0:1761315637.993042:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 6s 00000100:00080000:3.0:1761315637.993050:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761315637.993063:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761315637.993082:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000039277a44 x1846869307558848/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761315637.993103:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315637.993106:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761315637.993827:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307558848:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315637.994489:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761315637.994505:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307558784:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315637.996414:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000101f852a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307558720:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761315638.000866:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761315638.000883:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761315638.000895:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307558848:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315638.001397:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307558784:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315638.241259:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315638.308660:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315639.265515:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315639.329532:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315640.288828:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315640.354110:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315641.316825:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315641.377593:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315642.336819:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315642.402080:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761315643.106404:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315643.106414:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761315643.106420:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761315643.106434:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3886 00000100:00080000:2.0:1761315643.106439:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 11s 00000100:00080000:2.0:1761315643.106446:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761315643.106459:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761315643.106477:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000eca34e1c x1846869307559104/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761315643.106569:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eca34e1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307559104:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00080000:2.0:1761315643.107965:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315643.107991:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761315643.108163:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307558912:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315643.108234:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f220504f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307559040:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315643.108256:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761315643.108262:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b2298797 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307558976:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761315643.115732:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761315643.115744:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761315643.115755:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eca34e1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307559104:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315643.115931:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b2298797 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307558976:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315643.115943:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307558912:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315643.115953:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f220504f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307559040:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315643.361083:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315643.427617:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315644.385913:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315644.448951:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315645.411290:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315645.475866:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315646.434508:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315646.497292:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315647.456845:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315647.521193:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761315648.227262:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315648.227271:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761315648.227275:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761315648.227288:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3891 00000100:00080000:2.0:1761315648.227293:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 16s 00000100:00080000:2.0:1761315648.227300:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761315648.227310:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761315648.227326:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000039eebc2 x1846869307559360/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761315648.227350:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315648.227353:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761315648.227393:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307559232:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761315648.227525:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1761315648.227531:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f220504f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307559168:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761315648.227553:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b2298797 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307559296:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315648.227742:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000039eebc2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307559360:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761315648.233982:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f220504f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307559168:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761315648.241202:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307559232:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761315648.243171:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761315648.243184:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761315648.243214:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000039eebc2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307559360:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761315648.243374:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b2298797 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307559296:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315648.480819:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315648.546168:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315649.506049:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315649.569016:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315650.529179:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315650.594831:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315651.552787:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315651.616878:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315652.576855:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315652.640877:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761315653.349012:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315653.349028:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761315653.349033:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761315653.349049:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3896 00000100:00080000:3.0:1761315653.349055:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761315653.349063:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761315653.349076:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761315653.349089:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307559424:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315653.349101:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f133bb2e x1846869307559616/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761315653.349130:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315653.349153:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761315653.349187:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307559488:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315653.349209:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cc6dd8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307559552:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315653.352707:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f133bb2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307559616:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315653.359630:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307559424:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315653.359749:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc6dd8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307559552:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315653.359846:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307559488:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761315653.362222:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761315653.362238:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761315653.362250:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f133bb2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307559616:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761315653.600839:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315653.665786:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315654.628014:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315654.688914:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315655.649539:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315655.714517:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315656.676374:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315656.737420:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315657.696804:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315657.760843:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315658.464976:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b2298797 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307559680:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315658.465826:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761315658.465836:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307559744:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761315658.466595:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315658.466602:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761315658.466606:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761315658.466614:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3901 00000100:00080000:2.0:1761315658.466618:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761315658.466624:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761315658.466681:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761315658.466701:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e9376149 x1846869307559872/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761315658.466756:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315658.466760:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761315658.466783:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761315658.466797:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f220504f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307559808:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315658.466863:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e9376149 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307559872:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761315658.468193:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b2298797 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307559680:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315658.469952:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307559744:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315658.470579:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f220504f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307559808:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761315658.472196:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761315658.472206:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761315658.472216:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e9376149 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307559872:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761315658.720768:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315658.784917:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315659.745249:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315659.810799:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315660.769192:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315660.835011:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315661.792808:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315661.857689:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315662.818030:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315662.881076:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761315663.584852:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315663.584862:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761315663.584865:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761315663.584879:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3906 00000100:00080000:2.0:1761315663.584884:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1761315663.584892:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f220504f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307559936:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761315663.584894:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761315663.584909:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761315663.584929:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000039978a4e x1846869307560128/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761315663.584953:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315663.584957:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761315663.585004:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ae09c97d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307560000:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761315663.585019:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004592c0a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307560064:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315663.586331:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039978a4e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307560128:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761315663.589310:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f220504f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307559936:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761315663.594192:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761315663.594203:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761315663.594212:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039978a4e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307560128:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761315663.594956:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ae09c97d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307560000:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761315663.595009:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004592c0a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307560064:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315663.840751:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315663.906822:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315664.864774:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315664.929424:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315665.891590:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315665.953027:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315666.912775:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315666.977350:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315667.936781:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315668.001447:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761315668.706791:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315668.706798:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761315668.706815:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761315668.706825:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3911 00000100:00080000:2.0:1761315668.706829:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761315668.706835:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761315668.706845:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761315668.706862:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000011b5c99f x1846869307560384/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761315668.706896:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315668.706899:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761315668.706956:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000011b5c99f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307560384:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761315668.709111:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ae09c97d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307560192:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315668.709255:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000015f165e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307560320:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315668.709316:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761315668.709326:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e3466f5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307560256:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761315668.709436:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761315668.709446:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315668.709454:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000011b5c99f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307560384:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761315668.715469:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e3466f5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307560256:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315668.715521:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ae09c97d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307560192:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315668.715533:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000015f165e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307560320:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315668.961029:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315669.027303:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315669.984984:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315670.048819:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315671.008871:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315671.073561:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315672.033585:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315672.098486:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315673.057080:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315673.121086:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761315673.825320:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315673.825336:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761315673.825341:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761315673.825355:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3916 00000100:00100000:0.0:1761315673.825356:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307560448:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761315673.825359:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1761315673.825385:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761315673.825396:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761315673.825414:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000eb8b1f19 x1846869307560640/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761315673.825449:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315673.825453:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761315673.825458:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307560576:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761315673.825481:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307560640:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761315673.825490:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761315673.825498:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307560512:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315673.833536:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307560512:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315673.833566:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307560448:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315673.833576:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307560576:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:02020000:2.0:1761315673.833624:0:2262:0:(client.c:1361:ptlrpc_check_status()) 11-0: lustre-OST0000-osc-ffff951bc4d8c800: operation ost_connect to node 192.168.203.151@tcp failed: rc = -19 00000100:00080000:2.0:1761315673.848489:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761315673.848497:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315673.848509:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307560640:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761315674.081103:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315674.145231:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315675.104771:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315675.168898:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315676.129383:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315676.193598:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315677.152762:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315677.216886:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315678.176794:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315678.245063:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761315678.945396:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315678.945404:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761315678.945407:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761315678.945417:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3921 00000100:00080000:2.0:1761315678.945419:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761315678.945440:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761315678.945449:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761315678.945465:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008b3aaddc x1846869307560896/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761315678.945484:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315678.945488:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761315678.945603:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000011b5c99f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307560704:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761315678.945724:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ba5c8e77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307560832:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761315678.945754:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761315678.945759:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001426be00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307560768:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315678.946307:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b3aaddc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307560896:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761315678.952840:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001426be00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307560768:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761315678.952857:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000011b5c99f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307560704:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761315678.952866:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ba5c8e77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307560832:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761315678.953162:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761315678.953167:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761315678.953171:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b3aaddc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307560896:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761315679.201616:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315679.265220:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315680.224763:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315680.291946:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315681.248767:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315681.313128:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315682.274686:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315682.339093:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315683.296825:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315683.360916:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761315684.064896:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315684.064907:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761315684.064933:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761315684.064947:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3927 00000100:00080000:1.0:1761315684.064951:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1761315684.064959:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761315684.064972:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761315684.064993:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a15ef099 x1846869307561152/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761315684.065037:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315684.065041:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761315684.065076:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307561024:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761315684.065081:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307561152:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761315684.065227:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307560960:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315684.065697:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307561088:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315684.071044:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307561024:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315684.076875:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307560960:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315684.080261:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307561088:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761315684.080354:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761315684.080360:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315684.080365:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307561152:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761315684.320809:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315684.385770:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315685.347681:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315685.409983:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315686.374689:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315686.432900:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315687.394341:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315687.458429:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315688.416786:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315688.481882:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761315689.186720:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315689.186730:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761315689.186734:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761315689.186748:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3932 00000100:00080000:3.0:1761315689.186752:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761315689.186760:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761315689.186775:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761315689.186793:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ddd3c50a x1846869307561408/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761315689.187401:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315689.187429:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761315689.187814:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307561216:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315689.187925:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307561344:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315689.187965:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761315689.187972:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307561280:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315689.190119:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307561408:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315689.196992:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307561216:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315689.200818:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307561280:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315689.200847:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307561344:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761315689.200849:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761315689.200855:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761315689.200861:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307561408:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315689.442727:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315689.505539:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315690.464853:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315690.529504:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315691.490168:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315691.552906:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315692.515628:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315692.576925:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315693.536723:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315693.601430:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315694.305823:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ee08055d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307561472:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761315694.308152:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315694.308162:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761315694.308166:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761315694.308179:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3937 00000100:00080000:2.0:1761315694.308183:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761315694.308191:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761315694.308204:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761315694.308224:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e662bceb x1846869307561664/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761315694.308285:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e662bceb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307561664:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761315694.308308:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ffb03e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307561536:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315694.308412:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761315694.308419:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e1be9a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307561600:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315694.308788:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ee08055d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307561472:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761315694.309245:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315694.309253:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761315694.316061:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e1be9a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307561600:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315694.316077:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ffb03e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307561536:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761315694.316185:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761315694.316190:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761315694.316195:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e662bceb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307561664:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315694.560783:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315694.625521:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315695.584794:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315695.648969:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315696.609889:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315696.674657:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315697.632869:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315697.697491:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315698.656796:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315698.721715:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761315699.426951:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315699.426960:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761315699.426964:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761315699.426977:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3942 00000100:00080000:0.0:1761315699.426981:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761315699.427005:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761315699.427018:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761315699.427037:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000fe4020c4 x1846869307561920/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761315699.427067:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315699.427071:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761315699.427113:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307561792:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315699.427217:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1761315699.427224:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307561728:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315699.427249:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307561856:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761315699.431380:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307561920:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761315699.435957:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307561728:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315699.435988:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307561792:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315699.438228:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307561856:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761315699.440271:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761315699.440285:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761315699.440296:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307561920:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761315699.680787:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315699.745321:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315700.707179:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315700.769129:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315701.730084:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315701.794513:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315702.753181:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315702.817362:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315703.776763:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315703.840900:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761315704.544857:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315704.544865:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761315704.544869:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761315704.544881:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3947 00000100:00080000:3.0:1761315704.544885:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761315704.544892:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761315704.544903:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761315704.544919:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000004505177d x1846869307562176/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761315704.544957:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315704.544961:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761315704.544991:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004e00d332 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307562048:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315704.545107:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1761315704.545114:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307562176:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761315704.545117:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307561984:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315704.545157:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307562112:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315704.550126:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307562112:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315704.550153:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004e00d332 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307562048:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761315704.550627:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761315704.550634:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761315704.550689:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307562176:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761315704.553573:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307561984:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315704.800783:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315704.866677:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315705.825896:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315705.890139:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315706.848780:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315706.913991:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315707.873188:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315707.938352:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1761315707.938738:0:15001:0:(ldlm_pool.c:1444:ldlm_pools_recalc_task()) Negative interval(0) 00000100:00100000:2.0:1761315708.898528:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315708.961280:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761315709.664943:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315709.664953:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761315709.664957:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761315709.664970:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3952 00000100:00080000:3.0:1761315709.664974:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761315709.664982:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761315709.664995:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761315709.665016:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006c8fcfe5 x1846869307562432/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761315709.665110:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307562432:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761315709.665200:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307562240:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761315709.665243:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307562368:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761315709.665274:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761315709.665279:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004e00d332 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307562304:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761315709.666023:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315709.666029:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761315709.668322:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307562240:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761315709.668450:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761315709.668456:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761315709.668461:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307562432:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761315709.668679:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004e00d332 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307562304:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761315709.668692:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307562368:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761315709.922055:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315709.985840:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315710.944802:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315711.009955:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315711.968887:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315712.033254:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315712.992906:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315713.057451:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315714.016798:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315714.080901:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761315714.785053:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315714.785064:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761315714.785068:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1761315714.785069:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008a641e2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307562496:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761315714.785097:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3957 00000100:00080000:2.0:1761315714.785101:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761315714.785109:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761315714.785124:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761315714.785144:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000e1be9a7 x1846869307562688/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761315714.785220:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e1be9a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307562688:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761315714.786247:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ffb03e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307562624:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761315714.786249:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315714.786266:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761315714.786307:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e662bceb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307562560:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761315714.798798:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e662bceb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307562560:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761315714.798901:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008a641e2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307562496:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761315714.798915:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ffb03e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307562624:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761315714.803247:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761315714.803270:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761315714.803281:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e1be9a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307562688:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761315715.042681:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315715.107293:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315716.064807:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315716.128984:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315717.088821:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315717.152968:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315718.112863:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315718.177932:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315719.140221:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315719.200936:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761315719.904887:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315719.904893:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761315719.904896:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761315719.904906:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3962 00000100:00080000:0.0:1761315719.904909:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761315719.904915:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761315719.904923:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761315719.904939:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008e59d025 x1846869307562944/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1761315719.905013:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307562944:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00080000:0.0:1761315719.906096:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315719.906102:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761315719.906123:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307562816:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315719.906165:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1761315719.906170:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307562752:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315719.906184:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307562880:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315719.910941:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307562752:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315719.910992:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307562880:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315719.911006:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307562816:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761315719.914747:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761315719.914765:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761315719.914780:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307562944:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761315720.161026:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315720.224822:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315721.185096:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315721.249137:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315722.208867:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315722.272946:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315723.232792:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315723.296881:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315724.257977:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315724.325893:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761315725.025730:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315725.025740:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761315725.025744:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761315725.025758:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3967 00000100:00080000:3.0:1761315725.025762:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761315725.025768:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761315725.025779:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761315725.025800:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000d82b82f x1846869307563200/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761315725.025832:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315725.025835:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761315725.025869:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cdcc674d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307563072:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315725.026670:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307563200:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315725.026777:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000038534f0b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307563008:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315725.027829:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000626f98db pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307563136:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315725.032510:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cdcc674d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307563072:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315725.038303:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000626f98db pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307563136:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315725.038339:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000038534f0b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307563008:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761315725.038680:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761315725.038687:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761315725.038692:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307563200:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315725.280799:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315725.344920:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315726.306183:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315726.369099:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315727.331672:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315727.392920:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315728.352867:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315728.421398:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315729.376798:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315729.440940:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761315730.145536:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315730.145545:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761315730.145549:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761315730.145562:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3973 00000100:00080000:2.0:1761315730.145566:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761315730.145573:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761315730.145591:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761315730.145609:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ba5c8e77 x1846869307563456/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761315730.145682:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315730.145686:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761315730.145719:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ee08055d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307563328:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315730.146299:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1761315730.146310:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e662bceb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307563264:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315730.146352:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b3aaddc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307563392:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315730.147141:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ba5c8e77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307563456:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00080000:1.0:1761315730.153247:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761315730.153261:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761315730.153274:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ba5c8e77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307563456:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315730.154830:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e662bceb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307563264:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315730.154859:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b3aaddc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307563392:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315730.154900:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ee08055d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307563328:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315730.402761:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315730.465630:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315731.424841:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315731.488966:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315732.448786:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315732.516389:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315733.484550:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315733.537428:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315734.496758:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315734.566164:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315735.265662:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ee08055d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307563520:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761315735.268758:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315735.268769:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761315735.268773:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761315735.268784:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3978 00000100:00080000:2.0:1761315735.268788:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761315735.268793:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761315735.268802:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761315735.268817:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000011b5c99f x1846869307563712/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761315735.268843:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1761315735.268846:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e662bceb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307563648:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761315735.268848:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761315735.268874:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b3aaddc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307563584:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315735.268964:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000011b5c99f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307563712:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761315735.278408:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e662bceb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307563648:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315735.278423:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ee08055d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307563520:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315735.278529:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b3aaddc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307563584:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761315735.282122:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761315735.282133:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761315735.282141:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000011b5c99f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307563712:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761315735.520765:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315735.584885:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315736.544761:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315736.608889:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315737.571255:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315737.632844:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315738.592764:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315738.656845:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315739.618303:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315739.681076:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761315740.384969:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cdcc674d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307563776:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761315740.386039:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004e00d332 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307563840:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761315740.386390:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761315740.386398:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307563904:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761315740.387049:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315740.387054:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761315740.387057:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761315740.387062:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3983 00000100:00080000:3.0:1761315740.387065:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761315740.387069:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761315740.387085:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761315740.387101:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000602b09d0 x1846869307563968/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761315740.387124:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315740.387127:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761315740.387173:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307563968:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761315740.393116:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cdcc674d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307563776:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761315740.393241:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307563904:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761315740.393254:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004e00d332 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307563840:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761315740.396190:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761315740.396200:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761315740.396208:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307563968:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761315740.642527:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315740.705251:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315741.664713:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315741.728852:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315742.688961:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315742.753578:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315743.713303:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315743.777340:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315744.738965:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315744.800920:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761315745.505152:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b3aaddc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307564032:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761315745.505478:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315745.505484:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761315745.505488:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1761315745.505491:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001426be00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307564096:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761315745.505494:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3988 00000100:00080000:2.0:1761315745.505498:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761315745.505505:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761315745.505516:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761315745.505522:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761315745.505527:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000015f165e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307564160:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315745.505534:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ae09c97d x1846869307564224/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761315745.505600:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315745.505604:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761315745.505620:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ae09c97d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307564224:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315745.510608:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b3aaddc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307564032:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761315745.510754:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761315745.510764:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761315745.510769:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001426be00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307564096:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315745.510771:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ae09c97d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307564224:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761315745.510808:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000015f165e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307564160:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315745.760853:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315745.826157:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315746.784764:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315746.850113:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315747.809142:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315747.874578:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315748.834743:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315748.899345:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315749.856843:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315749.921345:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761315750.625678:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315750.625686:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761315750.625689:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761315750.625699:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3993 00000100:00080000:3.0:1761315750.625703:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761315750.625708:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761315750.625717:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761315750.625731:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ddd3c50a x1846869307564480/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761315750.625755:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315750.625758:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761315750.625784:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004e00d332 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307564288:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315750.625802:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307564480:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761315750.625868:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307564416:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315750.625898:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761315750.625902:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307564352:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315750.635226:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004e00d332 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307564288:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315750.635247:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307564416:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761315750.635547:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761315750.635552:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761315750.635557:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307564480:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761315750.636344:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307564352:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315750.880806:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315750.944899:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315751.904761:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315751.968964:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315752.928713:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315752.992940:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315753.952867:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315754.019814:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315754.976920:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315755.041062:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761315755.745073:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315755.745084:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761315755.745088:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761315755.745103:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 3998 00000100:00080000:2.0:1761315755.745107:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761315755.745115:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761315755.745128:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761315755.745279:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b3aaddc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307564544:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761315755.745517:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004592c0a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307564672:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761315755.745556:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761315755.745562:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e3466f5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307564608:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761315755.746032:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000039978a4e x1846869307564736/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761315755.746105:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039978a4e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307564736:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00080000:2.0:1761315755.747101:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315755.747137:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761315755.753610:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e3466f5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307564608:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761315755.753633:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b3aaddc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307564544:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761315755.753669:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004592c0a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307564672:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761315755.754346:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761315755.754352:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761315755.754358:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039978a4e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307564736:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761315756.000843:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315756.064905:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315757.024834:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315757.091055:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315758.050726:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315758.117845:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315759.072815:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315759.138016:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315760.097139:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315760.160943:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761315760.865034:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315760.865043:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761315760.865047:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761315760.865060:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4003 00000100:00080000:0.0:1761315760.865064:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761315760.865070:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761315760.865094:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761315760.865137:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000bcaee194 x1846869307564992/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761315760.865164:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315760.865172:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761315760.865206:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307564864:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315760.865206:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307564992:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761315760.865276:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307564800:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315760.865318:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307564928:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315760.873429:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307564864:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315760.873442:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307564800:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315760.873459:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307564928:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761315760.875136:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761315760.875150:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761315760.875160:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307564992:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761315761.121124:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315761.185837:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315762.145588:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315762.209333:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315763.168835:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315763.236813:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315764.192785:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315764.257757:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315765.216842:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315765.281420:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761315765.985034:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315765.985043:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761315765.985047:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761315765.985060:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4008 00000100:00080000:0.0:1761315765.985064:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761315765.985072:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761315765.985086:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761315765.985107:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000005b900d12 x1846869307565248/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761315765.985139:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315765.985161:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761315765.985222:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307565120:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315765.985337:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1761315765.985344:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307565056:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315765.985372:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307565184:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315765.985869:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307565248:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00080000:2.0:1761315766.002428:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761315766.002443:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315766.002456:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307565248:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761315766.003333:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307565056:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315766.003377:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307565184:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315766.003389:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307565120:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315766.240803:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315766.305121:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315767.264726:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315767.329299:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315768.288826:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315768.352943:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315769.315217:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315769.377795:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315770.336812:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315770.401049:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761315771.104864:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315771.104871:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761315771.104875:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761315771.104886:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4014 00000100:00080000:0.0:1761315771.104890:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761315771.104896:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000100:00100000:1.0:1761315771.104903:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307565312:192.168.203.151@tcp:400:kworker/u8:1.0 00000008:00100000:0.0:1761315771.104907:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761315771.104927:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e00b5e9d x1846869307565504/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761315771.104956:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315771.104959:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761315771.104993:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307565440:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315771.105027:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761315771.105033:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307565376:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315771.105983:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307565504:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761315771.111710:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307565312:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315771.112571:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307565376:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315771.112601:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307565440:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761315771.113285:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761315771.113291:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315771.113298:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307565504:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761315771.360898:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315771.425561:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315772.384758:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315772.448890:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315773.409567:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315773.475855:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315774.434320:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315774.496954:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315775.460181:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315775.526754:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761315776.226256:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315776.226266:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761315776.226271:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761315776.226285:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4019 00000100:00080000:3.0:1761315776.226309:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761315776.226318:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761315776.226332:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761315776.226353:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e6d9e11e x1846869307565760/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761315776.226377:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315776.226381:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761315776.226423:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307565632:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761315776.226550:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1761315776.226559:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307565568:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761315776.226585:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004e00d332 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307565696:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315776.226917:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307565760:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761315776.229854:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004e00d332 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307565696:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761315776.229876:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307565632:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761315776.234872:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307565568:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761315776.236962:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761315776.236976:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761315776.236986:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307565760:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761315776.481399:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315776.545341:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315777.505034:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315777.568892:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315778.528781:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315778.595224:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315779.552804:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315779.618957:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315780.576874:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315780.642425:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761315781.348284:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315781.348293:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761315781.348297:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761315781.348310:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4024 00000100:00080000:2.0:1761315781.348314:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761315781.348321:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761315781.348333:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761315781.348351:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b2298797 x1846869307566016/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761315781.348377:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315781.348381:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761315781.348411:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e9376149 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307565888:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315781.348551:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1761315781.348571:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f220504f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307565824:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315781.348602:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307565952:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315781.348721:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b2298797 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307566016:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00080000:0.0:1761315781.365177:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761315781.365192:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761315781.365205:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b2298797 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307566016:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315781.366114:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f220504f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307565824:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315781.366135:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307565952:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315781.366146:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e9376149 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307565888:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315781.600734:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315781.671315:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315782.624808:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315782.691455:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315783.648881:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315783.714274:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315784.673919:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315784.736886:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315785.697060:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315785.762281:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315786.467928:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307566080:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761315786.468713:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315786.468719:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761315786.468723:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761315786.468729:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4029 00000100:00080000:0.0:1761315786.468733:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761315786.468739:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761315786.468759:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761315786.468776:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007dcceb4a x1846869307566272/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761315786.468799:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315786.468803:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761315786.468861:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307566208:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315786.468899:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761315786.468903:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307566144:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315786.469488:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307566272:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761315786.473888:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307566080:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761315786.475190:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761315786.475199:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315786.475208:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307566272:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761315786.475528:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307566144:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315786.475540:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307566208:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315786.720876:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315786.785089:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315787.744864:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315787.809298:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315788.770502:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315788.832898:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315789.793375:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315789.858686:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315790.817878:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315790.880907:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761315791.586101:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315791.586109:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761315791.586112:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761315791.586124:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4034 00000100:00080000:3.0:1761315791.586128:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761315791.586134:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761315791.586145:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761315791.586176:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000bfffbafe x1846869307566528/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761315791.586193:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315791.586196:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761315791.586225:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307566400:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761315791.586322:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1761315791.586328:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307566336:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761315791.586352:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004e00d332 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307566464:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315791.588365:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307566528:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761315791.595299:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307566400:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761315791.595333:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307566336:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761315791.595837:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761315791.595842:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761315791.595857:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307566528:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761315791.601596:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004e00d332 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307566464:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315791.840857:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315791.906407:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315792.864801:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315792.928909:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315793.888786:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315793.952918:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315794.916864:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315794.982031:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315795.936832:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315796.003841:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761315796.705736:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315796.705745:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761315796.705749:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761315796.705761:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4039 00000100:00080000:2.0:1761315796.705765:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761315796.705771:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761315796.705783:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761315796.705801:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000039eebc2 x1846869307566784/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761315796.705874:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000039eebc2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307566784:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761315796.706503:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e9376149 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307566592:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315796.706559:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f220504f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307566720:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315796.706592:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761315796.706600:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307566656:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761315796.708574:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315796.708606:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00080000:0.0:1761315796.709454:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761315796.709464:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761315796.709476:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000039eebc2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307566784:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761315796.715276:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e9376149 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307566592:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315796.717582:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307566656:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315796.717605:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f220504f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307566720:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315796.962380:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315797.026064:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315797.984751:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315798.049449:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315799.009591:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315799.073990:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315800.033285:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315800.098699:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315801.056777:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315801.124449:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761315801.825268:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315801.825275:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761315801.825278:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761315801.825288:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4044 00000100:00080000:0.0:1761315801.825291:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761315801.825297:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761315801.825307:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761315801.825322:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000057b18867 x1846869307567040/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761315801.825362:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315801.825365:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761315801.825386:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307566912:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315801.825470:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1761315801.825476:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307566848:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315801.825499:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307566976:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315801.825865:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307567040:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00080000:2.0:1761315801.831298:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761315801.831308:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315801.831317:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307567040:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761315801.831736:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307566848:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315801.831773:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307566976:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315801.831787:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307566912:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315802.101588:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315802.147402:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315803.104844:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315803.169160:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315804.128838:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315804.196316:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315805.152848:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315805.216889:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315806.176796:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315806.243008:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761315806.945037:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315806.945057:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761315806.945061:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761315806.945075:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4049 00000100:00080000:3.0:1761315806.945079:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761315806.945086:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761315806.945098:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761315806.945115:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a183f7ef x1846869307567296/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761315806.945142:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315806.945146:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761315806.945168:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307567168:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761315806.945280:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1761315806.945287:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004e00d332 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307567104:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761315806.945313:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307567232:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315806.945791:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307567296:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761315806.952155:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004e00d332 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307567104:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761315806.952183:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307567232:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761315806.952193:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307567168:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761315806.955295:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761315806.955310:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761315806.955322:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307567296:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761315807.200794:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315807.276521:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315808.226656:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315808.288921:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315809.248893:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315809.313946:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315810.272791:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315810.339148:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315811.296857:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315811.361777:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761315812.065625:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315812.065691:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761315812.065696:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761315812.065711:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4055 00000100:00080000:1.0:1761315812.065715:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761315812.065722:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761315812.065735:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761315812.065758:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a15ef099 x1846869307567552/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761315812.065776:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315812.065779:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761315812.065817:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307567424:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315812.065942:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1761315812.065949:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307567360:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315812.065975:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307567488:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761315812.066262:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307567552:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00080000:2.0:1761315812.073589:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761315812.073603:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315812.073616:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307567552:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761315812.073747:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307567360:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315812.073764:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307567488:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315812.073799:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307567424:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315812.321129:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315812.390853:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315813.344733:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315813.409168:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315814.368800:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315814.435048:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315815.394131:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315815.456898:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315816.416751:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315816.480903:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761315817.187968:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315817.187977:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761315817.187981:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761315817.187992:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4060 00000100:00080000:2.0:1761315817.187996:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761315817.188003:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000100:00100000:3.0:1761315817.188007:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eca34e1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307567616:192.168.203.151@tcp:400:kworker/u8:1.0 00000008:00100000:2.0:1761315817.188013:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761315817.188031:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000aea3302e x1846869307567808/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761315817.188060:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315817.188064:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761315817.188087:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000007136630 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307567680:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315817.188100:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002292ab57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307567744:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315817.188117:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aea3302e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307567808:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761315817.192324:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eca34e1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307567616:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315817.192928:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000007136630 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307567680:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315817.193002:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002292ab57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307567744:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761315817.193151:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761315817.193157:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761315817.193163:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aea3302e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307567808:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761315817.442398:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315817.506850:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315818.464788:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315818.529957:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315819.501631:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315819.552857:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315820.512871:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315820.576917:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315821.536807:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315821.600921:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761315822.305093:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315822.305103:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761315822.305106:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761315822.305119:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4065 00000100:00080000:2.0:1761315822.305123:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761315822.305129:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761315822.305142:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761315822.305161:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000002292ab57 x1846869307568064/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761315822.305186:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315822.305189:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761315822.305229:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000007136630 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307567872:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315822.305239:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b4eac89a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307567936:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315822.305355:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aea3302e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307568000:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315822.305457:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002292ab57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307568064:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00080000:0.0:1761315822.311773:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761315822.311789:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761315822.311802:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002292ab57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307568064:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315822.311925:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b4eac89a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307567936:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315822.313251:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000007136630 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307567872:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315822.313270:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aea3302e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307568000:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315822.560797:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315822.626929:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315823.584739:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315823.651047:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315824.608802:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315824.673451:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315825.632915:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315825.697902:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315826.656768:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315826.721687:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761315827.430781:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315827.430791:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761315827.430795:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1761315827.430802:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307568128:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761315827.430807:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4070 00000100:00080000:3.0:1761315827.430812:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761315827.430819:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761315827.430832:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761315827.430851:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f133bb2e x1846869307568320/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761315827.430876:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315827.430880:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761315827.430908:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307568192:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761315827.430915:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004e00d332 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307568256:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315827.430984:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f133bb2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307568320:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:02020000:0.0:1761315827.438718:0:2262:0:(client.c:1361:ptlrpc_check_status()) 11-0: lustre-OST0000-osc-ffff951bc4d8c800: operation ost_connect to node 192.168.203.151@tcp failed: rc = -19 00000100:00080000:0.0:1761315827.438729:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761315827.438735:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761315827.438740:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f133bb2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307568320:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761315827.438772:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307568192:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761315827.439219:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307568128:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761315827.439241:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004e00d332 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307568256:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315827.684697:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315827.746270:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315828.704840:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315828.768958:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315829.728776:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315829.793372:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315830.752747:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315830.817895:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315831.776770:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315831.844814:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761315832.544843:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315832.544851:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761315832.544853:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761315832.544863:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4075 00000100:00080000:3.0:1761315832.544866:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761315832.544873:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761315832.544882:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761315832.544896:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000004c6aebc5 x1846869307568576/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761315832.544921:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315832.544924:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761315832.544968:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307568576:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761315832.545848:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307568384:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761315832.545914:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307568512:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761315832.545939:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761315832.545945:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cc6dd8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307568448:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761315832.556695:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc6dd8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307568448:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761315832.556715:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307568512:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761315832.558025:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761315832.558032:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761315832.558040:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307568576:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761315832.558139:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307568384:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315832.800784:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315832.864878:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315833.824993:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315833.891479:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315834.849219:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315834.912903:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315835.872797:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315835.947547:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315836.897145:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315836.960880:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761315837.664917:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315837.664926:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761315837.664931:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761315837.664945:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4080 00000100:00080000:0.0:1761315837.664949:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761315837.664978:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000100:00100000:1.0:1761315837.664981:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307568640:192.168.203.151@tcp:400:kworker/u8:3.0 00000008:00100000:0.0:1761315837.664994:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761315837.665020:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000bcaee194 x1846869307568832/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761315837.665050:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315837.665056:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761315837.665090:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307568704:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761315837.665111:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307568832:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761315837.665864:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307568768:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315837.675591:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307568704:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315837.675837:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307568768:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315837.675857:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307568640:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761315837.681366:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761315837.681381:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315837.681402:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307568832:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761315837.921217:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315837.984865:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315838.944789:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315839.011529:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315839.972529:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315840.043978:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315840.992801:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315841.056914:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315842.016801:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315842.080941:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761315842.784851:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315842.784858:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761315842.784860:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761315842.784871:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4085 00000100:00080000:0.0:1761315842.784874:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761315842.784880:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761315842.784889:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761315842.784904:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008e59d025 x1846869307569088/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761315842.784927:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315842.784930:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761315842.784956:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307568960:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315842.785018:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307569088:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761315842.785068:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1761315842.785080:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307568896:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315842.785123:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307569024:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315842.795111:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307568896:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315842.795132:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307569024:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315842.795141:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307568960:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761315842.796949:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761315842.796962:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315842.796975:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307569088:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761315843.042665:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315843.106139:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315844.065686:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315844.128945:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315845.088988:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315845.152865:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315846.113819:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315846.180598:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315847.136804:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315847.200870:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761315847.906769:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315847.906778:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761315847.906782:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761315847.906794:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4090 00000100:00080000:3.0:1761315847.906797:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761315847.906804:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761315847.906815:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761315847.906829:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000bc14d8e x1846869307569344/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761315847.906861:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315847.906877:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761315847.906977:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307569344:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761315847.908374:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000101f852a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307569152:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761315847.909459:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d32bed84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307569216:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761315847.913966:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f597cda6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307569280:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761315847.915195:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761315847.915205:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315847.915226:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307569344:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761315847.915565:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d32bed84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307569216:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761315847.920351:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000101f852a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307569152:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761315847.920372:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f597cda6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307569280:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315848.160765:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315848.225349:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315849.184754:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315849.248834:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315850.212467:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315850.273093:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315851.232762:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315851.296892:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315852.256778:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315852.320851:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761315853.026985:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315853.026991:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761315853.026995:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761315853.027005:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4095 00000100:00080000:0.0:1761315853.027008:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761315853.027014:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761315853.027023:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761315853.027035:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007a68d9a8 x1846869307569600/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761315853.027057:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315853.027060:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761315853.027139:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307569408:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315853.027221:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307569536:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315853.027248:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761315853.027252:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307569472:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761315853.027343:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307569600:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761315853.031421:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307569408:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315853.033310:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307569536:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315853.039408:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307569472:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761315853.040171:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761315853.040180:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761315853.040187:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307569600:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761315853.283629:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315853.347954:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315854.304773:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315854.370466:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315855.328795:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315855.392981:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315856.352817:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315856.419118:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315857.376794:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315857.441174:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315858.145515:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307569664:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315858.149114:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761315858.149136:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307569728:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761315858.149900:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315858.149908:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761315858.149932:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761315858.149939:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4101 00000100:00080000:0.0:1761315858.149943:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761315858.149950:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761315858.149963:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761315858.149982:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a24c5c6a x1846869307569856/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761315858.150005:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315858.150009:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761315858.150029:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761315858.150040:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307569792:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315858.150220:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307569856:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761315858.158523:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307569664:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761315858.161793:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:0.0:1761315858.161800:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307569792:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761315858.161805:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761315858.161814:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307569856:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761315858.161922:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307569728:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315858.401151:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315858.464930:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315859.424811:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315859.489411:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315860.448780:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315860.512927:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315861.472800:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315861.538709:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315862.497565:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315862.561762:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761315863.264873:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315863.264883:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761315863.264887:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761315863.264901:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4106 00000100:00080000:0.0:1761315863.264904:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761315863.264912:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761315863.264925:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761315863.264943:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a24c5c6a x1846869307570112/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761315863.264972:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315863.264975:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761315863.265096:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307569920:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315863.265214:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307570048:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315863.265250:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761315863.265254:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307569984:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315863.265581:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307570112:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761315863.268618:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307569984:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315863.268862:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307569920:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315863.268875:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307570048:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761315863.269238:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761315863.269244:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761315863.269249:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307570112:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761315863.521242:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315863.585670:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315864.544781:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315864.608936:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315865.568799:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315865.632886:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315866.592802:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315866.657664:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315867.617562:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315867.681201:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761315868.386631:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315868.386685:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761315868.386690:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761315868.386703:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4111 00000100:00080000:1.0:1761315868.386707:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761315868.386715:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761315868.386727:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761315868.386746:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000009cec3faa x1846869307570368/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761315868.386955:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307570176:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315868.387243:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307570304:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315868.387301:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761315868.387307:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307570240:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761315868.387603:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315868.387607:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761315868.387671:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307570368:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00080000:3.0:1761315868.395234:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761315868.395249:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761315868.395262:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307570368:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761315868.398431:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307570240:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315868.398456:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307570176:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315868.398468:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307570304:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315868.641030:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315868.706059:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315869.664807:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315869.729266:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315870.688840:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315870.752891:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315871.712958:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315871.777404:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315872.736822:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315872.801942:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761315873.505785:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315873.505794:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761315873.505798:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761315873.505814:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4116 00000100:00080000:1.0:1761315873.505818:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761315873.505826:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761315873.505838:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761315873.505858:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000163e3801 x1846869307570624/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1761315873.505938:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307570624:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00080000:1.0:1761315873.506447:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315873.506453:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761315873.506486:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307570496:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315873.506553:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1761315873.506560:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307570432:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315873.506592:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307570560:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761315873.514695:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761315873.514711:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315873.514723:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307570624:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761315873.514979:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307570432:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315873.514998:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307570560:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315873.515034:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307570496:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315873.764396:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315873.824933:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315874.784772:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315874.849375:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315875.808925:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315875.872926:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315876.833939:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315876.897855:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315877.856817:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315877.921827:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761315878.624933:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315878.624942:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761315878.624947:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761315878.624959:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4121 00000100:00080000:1.0:1761315878.624963:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761315878.624970:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761315878.624982:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761315878.624999:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000540c3ce6 x1846869307570880/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761315878.625039:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315878.625043:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761315878.625241:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307570880:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761315878.625437:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307570688:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315878.625474:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307570816:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315878.625501:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761315878.625507:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307570752:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315878.632666:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307570752:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315878.632692:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307570816:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761315878.632878:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761315878.632883:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315878.632888:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307570880:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761315878.634700:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307570688:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315878.880808:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315878.946874:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315879.904795:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315879.968913:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315880.928796:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315880.994374:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315881.952838:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315882.016905:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315882.977323:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315883.040828:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761315883.745561:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315883.745573:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761315883.745578:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761315883.745593:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4126 00000100:00080000:0.0:1761315883.745597:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761315883.745606:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761315883.745620:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761315883.745713:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000028de4b9d x1846869307571136/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761315883.745741:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315883.745745:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761315883.745796:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307571136:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761315883.745828:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307571008:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315883.745911:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1761315883.745919:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307570944:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315883.745948:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307571072:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315883.749707:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307570944:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315883.749730:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307571072:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315883.749740:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307571008:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761315883.749878:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761315883.749884:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761315883.749888:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307571136:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761315884.000828:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315884.067518:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315885.024734:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315885.090258:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315886.048819:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315886.114057:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315887.072805:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315887.136917:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315888.096805:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315888.164214:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761315888.864987:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315888.864995:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761315888.864999:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761315888.865009:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4131 00000100:00080000:2.0:1761315888.865012:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761315888.865018:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761315888.865057:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761315888.865072:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b8c05a37 x1846869307571392/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761315888.865157:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307571392:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00080000:2.0:1761315888.866072:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315888.866077:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761315888.866111:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eca34e1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307571264:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315888.866869:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b4eac89a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307571200:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315888.867521:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f220504f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307571328:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315888.870601:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b4eac89a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307571200:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315888.870631:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f220504f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307571328:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761315888.871119:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761315888.871143:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761315888.871149:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307571392:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315888.871264:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eca34e1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307571264:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315889.120844:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315889.186134:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315890.145141:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315890.209371:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315891.168979:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315891.232916:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315892.192812:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315892.256903:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315893.216798:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315893.280944:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761315893.998502:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315893.998511:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761315893.998515:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761315893.998526:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4136 00000100:00100000:3.0:1761315893.998527:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e9376149 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307571520:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761315893.998531:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761315893.998539:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761315893.998553:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761315893.998572:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b2298797 x1846869307571648/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761315893.998600:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315893.998604:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761315893.998635:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1761315893.998672:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eca34e1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307571456:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315893.998695:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000039eebc2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307571584:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315893.999289:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b2298797 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307571648:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761315894.004460:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000039eebc2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307571584:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315894.004482:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e9376149 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307571520:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315894.004690:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eca34e1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307571456:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761315894.010106:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761315894.010119:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761315894.010130:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b2298797 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307571648:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761315894.240785:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315894.304863:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315895.264779:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315895.328865:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315896.291065:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315896.362119:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315897.314181:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315897.376864:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315898.337047:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315898.401152:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761315899.106925:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315899.106932:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761315899.106936:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761315899.106946:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4142 00000100:00080000:2.0:1761315899.106959:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761315899.106966:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761315899.106976:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761315899.107003:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006e3466f5 x1846869307571904/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761315899.107094:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e3466f5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307571904:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00080000:2.0:1761315899.107356:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315899.107361:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761315899.107378:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004592c0a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307571776:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315899.107424:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1761315899.107430:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039978a4e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307571712:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315899.107448:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b3aaddc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307571840:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761315899.110992:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761315899.111004:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761315899.111015:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e3466f5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307571904:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315899.111114:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039978a4e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307571712:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315899.111130:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b3aaddc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307571840:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315899.111140:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004592c0a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307571776:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315899.361691:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315899.425062:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315900.386757:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315900.451796:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315901.408993:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315901.475311:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315902.432798:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315902.498768:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315903.456801:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315903.520999:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761315904.224985:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315904.224995:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761315904.224999:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761315904.225013:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4147 00000100:00080000:0.0:1761315904.225018:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761315904.225026:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761315904.225038:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761315904.225058:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000005b900d12 x1846869307572160/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1761315904.225133:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307572160:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761315904.225300:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307571968:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315904.225330:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307572096:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315904.225354:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761315904.225358:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307572032:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761315904.226770:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315904.226782:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00080000:3.0:1761315904.233189:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761315904.233205:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761315904.233218:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307572160:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761315904.233447:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307572032:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315904.233460:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307572096:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315904.233850:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307571968:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315904.481486:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315904.544971:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315905.504774:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315905.570879:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315906.528822:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315906.594519:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315907.552804:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315907.616973:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315908.576804:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315908.640929:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761315909.347024:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315909.347032:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761315909.347037:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761315909.347051:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4152 00000100:00080000:1.0:1761315909.347064:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761315909.347071:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761315909.347083:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761315909.347102:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ed2f9bca x1846869307572416/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761315909.347124:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315909.347128:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761315909.347189:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307572416:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761315909.347381:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307572288:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315909.348123:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1761315909.348135:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307572224:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315909.348743:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307572352:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761315909.359711:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761315909.359727:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761315909.359756:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307572224:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761315909.359767:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307572416:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761315909.359780:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307572352:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315909.359796:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307572288:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315909.600869:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315909.664913:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315910.624795:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315910.689540:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315911.648799:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315911.713173:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315912.672801:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315912.736906:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315913.697571:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315913.762542:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761315914.466594:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315914.466602:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761315914.466606:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761315914.466618:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4157 00000100:00080000:3.0:1761315914.466622:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761315914.466629:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761315914.466680:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761315914.466700:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007673f690 x1846869307572672/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761315914.466726:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315914.466730:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761315914.466765:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f364ec74 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307572544:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315914.466896:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1761315914.466902:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d32bed84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307572480:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315914.466927:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307572608:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761315914.468289:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007673f690 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307572672:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761315914.469194:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307572608:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315914.469213:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f364ec74 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307572544:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315914.469467:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d32bed84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307572480:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761315914.472753:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761315914.472787:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761315914.472801:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007673f690 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307572672:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761315914.727011:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315914.787449:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315915.744812:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315915.809364:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315916.769142:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315916.833530:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761315917.793433:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315917.858532:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315918.826516:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315918.880893:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761315919.584866:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315919.584873:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761315919.584876:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761315919.584886:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4162 00000100:00080000:3.0:1761315919.584890:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761315919.584901:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761315919.584912:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761315919.584929:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006e93398e x1846869307572928/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761315919.584959:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315919.584962:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761315919.585061:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307572928:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315919.585282:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007673f690 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307572736:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315919.585319:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f364ec74 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307572864:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315919.585343:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761315919.585348:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d32bed84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307572800:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761315919.588303:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761315919.588320:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761315919.588332:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307572928:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315919.588442:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d32bed84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307572800:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315919.588456:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007673f690 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307572736:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315919.589289:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f364ec74 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307572864:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315919.843657:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315919.905015:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315920.864780:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315920.929122:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315921.888860:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315921.953376:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315922.912802:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315922.976940:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315923.936810:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315924.000948:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761315924.704992:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315924.705002:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761315924.705007:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761315924.705020:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4167 00000100:00080000:1.0:1761315924.705024:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761315924.705032:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761315924.705046:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761315924.705067:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a15ef099 x1846869307573184/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1761315924.705140:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307573184:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761315924.705710:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307572992:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315924.705753:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307573120:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315924.705802:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761315924.705810:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307573056:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761315924.706341:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315924.706346:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761315924.710904:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307572992:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761315924.717417:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761315924.717431:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761315924.718669:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307573184:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761315924.718808:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307573056:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315924.718826:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307573120:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315924.961096:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315925.024954:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315925.984829:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315926.048905:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315927.008814:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315927.076236:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315928.033576:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315928.100009:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315929.056818:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315929.122160:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761315929.825038:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315929.825049:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761315929.825053:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761315929.825068:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4172 00000100:00080000:0.0:1761315929.825071:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761315929.825078:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761315929.825092:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761315929.825110:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000094820351 x1846869307573440/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761315929.825135:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315929.825139:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761315929.825171:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307573312:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761315929.825185:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307573440:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761315929.825301:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1761315929.825309:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307573248:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315929.825351:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307573376:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761315929.833936:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761315929.833950:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761315929.833961:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307573440:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761315929.834260:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307573312:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315929.834683:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307573248:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315929.834696:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307573376:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315930.080807:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315930.144933:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315931.104820:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315931.170032:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315932.129250:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315932.193512:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315933.152944:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315933.218329:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315934.176793:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315934.243101:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761315934.945544:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315934.945570:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761315934.945574:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761315934.945586:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4177 00000100:00080000:2.0:1761315934.945591:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761315934.945599:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761315934.945613:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761315934.945634:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000015f165e6 x1846869307573696/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1761315934.945823:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004592c0a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307573504:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315934.945926:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039978a4e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307573632:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315934.945956:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761315934.945960:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b3aaddc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307573568:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761315934.946780:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315934.946784:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761315934.946917:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000015f165e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307573696:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761315934.956857:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004592c0a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307573504:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761315934.957171:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761315934.957177:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761315934.957182:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000015f165e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307573696:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761315934.957263:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b3aaddc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307573568:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315934.957274:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039978a4e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307573632:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315935.200885:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315935.266131:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315936.225094:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315936.288927:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315937.249418:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315937.313327:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315938.272807:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315938.337385:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315939.297735:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315939.360964:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761315940.067950:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315940.067960:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761315940.067964:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761315940.067977:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4183 00000100:00080000:2.0:1761315940.067981:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761315940.067988:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761315940.068001:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761315940.068020:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ee08055d x1846869307573952/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761315940.068043:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315940.068046:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761315940.068081:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ae09c97d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307573824:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315940.068240:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1761315940.068247:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001426be00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307573760:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315940.068274:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000011b5c99f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307573888:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315940.068753:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ee08055d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307573952:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315940.070089:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001426be00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307573760:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315940.070112:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000011b5c99f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307573888:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315940.070133:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ae09c97d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307573824:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761315940.073078:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761315940.073095:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761315940.073108:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ee08055d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307573952:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761315940.320806:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315940.388317:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315941.344837:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315941.409043:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315942.368807:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315942.432935:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315943.393217:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315943.458219:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315944.416814:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315944.481168:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761315945.185060:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315945.185070:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761315945.185075:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761315945.185088:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4188 00000100:00080000:3.0:1761315945.185093:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761315945.185100:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761315945.185118:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761315945.185137:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000041c8b6cb x1846869307574208/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761315945.185183:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315945.185187:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761315945.185221:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e58cc722 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307574080:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315945.185336:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1761315945.185342:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307574016:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315945.185367:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000009911ad3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307574144:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315945.185498:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000041c8b6cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307574208:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00080000:0.0:1761315945.195456:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761315945.195470:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761315945.195482:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000041c8b6cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307574208:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761315945.195570:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307574016:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315945.195586:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000009911ad3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307574144:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315945.195597:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e58cc722 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307574080:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315945.445889:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315945.504919:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315946.464802:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315946.530264:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315947.488998:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315947.552894:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315948.512816:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315948.578101:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315949.536734:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315949.601232:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761315950.305213:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315950.305223:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761315950.305226:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761315950.305240:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4193 00000100:00080000:0.0:1761315950.305244:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00100000:1.0:1761315950.305246:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307574272:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761315950.305251:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761315950.305264:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761315950.305292:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e00b5e9d x1846869307574464/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761315950.305336:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315950.305340:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761315950.305351:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307574400:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315950.305389:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307574336:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761315950.305636:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307574464:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761315950.309384:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307574336:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315950.311661:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307574272:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315950.311680:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307574400:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761315950.312225:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761315950.312231:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761315950.312237:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307574464:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761315950.560800:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315950.624858:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315951.586517:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315951.649012:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315952.608794:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315952.675528:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315953.632813:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761315953.697678:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315954.656812:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315954.722271:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761315955.425108:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315955.425118:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761315955.425123:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761315955.425136:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4198 00000100:00080000:2.0:1761315955.425140:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761315955.425147:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761315955.425160:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761315955.425179:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e662bceb x1846869307574720/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761315955.425209:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315955.425213:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761315955.425247:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000011b5c99f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307574592:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315955.425384:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1761315955.425392:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ae09c97d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307574528:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315955.425444:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001426be00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307574656:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315955.425908:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e662bceb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307574720:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315955.431352:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ae09c97d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307574528:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315955.431384:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001426be00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307574656:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315955.431398:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000011b5c99f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307574592:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761315955.437999:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761315955.438014:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761315955.438026:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e662bceb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307574720:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761315955.680875:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315955.744923:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315956.704819:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315956.770001:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315957.729526:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315957.792912:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315958.754800:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315958.816886:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315959.776852:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315959.841725:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761315960.544884:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315960.544894:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761315960.544898:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761315960.544911:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4203 00000100:00100000:0.0:1761315960.544914:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307574784:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761315960.544915:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761315960.544921:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761315960.544933:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761315960.544969:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a15ef099 x1846869307574976/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761315960.544999:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315960.545002:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761315960.545019:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307574912:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315960.545054:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761315960.545059:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307574848:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761315960.545121:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307574976:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761315960.549316:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307574848:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315960.549341:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307574912:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761315960.549829:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761315960.549835:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761315960.549860:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307574976:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761315960.553730:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307574784:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315960.800871:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315960.867807:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315961.824841:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315961.890045:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315962.848834:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315962.913560:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315963.873304:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315963.936932:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315964.896836:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315964.961874:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761315965.665109:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315965.665118:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761315965.665122:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761315965.665135:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4208 00000100:00080000:0.0:1761315965.665139:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761315965.665145:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761315965.665165:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761315965.665184:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b5a39135 x1846869307575232/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761315965.665240:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315965.665244:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761315965.665281:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307575104:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761315965.665378:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307575232:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761315965.665546:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307575040:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315965.666732:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307575168:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761315965.680969:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761315965.680984:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315965.680998:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307575232:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761315965.681196:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307575040:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315965.681212:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307575168:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315965.681261:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307575104:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315965.920794:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315965.985421:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315966.944860:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315967.008929:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315967.970610:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315968.033073:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315968.992995:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315969.056911:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315970.016839:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315970.081927:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761315970.785056:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315970.785064:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761315970.785068:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761315970.785080:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4213 00000100:00080000:1.0:1761315970.785084:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761315970.785091:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761315970.785102:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761315970.785138:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ed2f9bca x1846869307575488/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761315970.785212:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307575488:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761315970.785348:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307575296:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315970.785376:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307575424:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315970.785401:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761315970.785406:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307575360:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761315970.789142:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761315970.789156:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761315970.793940:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307575360:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315970.793963:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307575296:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315970.793974:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307575424:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761315970.794079:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761315970.794086:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315970.794091:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307575488:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761315971.042565:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315971.105721:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315972.064810:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315972.129119:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315973.088792:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315973.157126:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315974.112832:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315974.176922:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315975.136796:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315975.200931:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761315975.905148:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315975.905157:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761315975.905161:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761315975.905175:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4218 00000100:00080000:3.0:1761315975.905179:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761315975.905186:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761315975.905197:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761315975.905217:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000041c8b6cb x1846869307575744/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761315975.905298:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000041c8b6cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307575744:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00080000:3.0:1761315975.905928:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315975.905934:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761315975.905961:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000009911ad3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307575616:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761315975.906015:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1761315975.906022:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e58cc722 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307575552:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761315975.906046:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307575680:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761315975.908851:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e58cc722 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307575552:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761315975.910031:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761315975.910040:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761315975.910048:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000041c8b6cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307575744:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761315975.910216:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307575680:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761315975.910228:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000009911ad3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307575616:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315976.160801:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315976.225467:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315977.185464:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315977.249499:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315978.208788:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315978.277514:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315979.232787:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315979.296917:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315980.256795:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315980.321463:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761315981.025274:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315981.025301:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761315981.025305:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1761315981.025309:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000009911ad3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307575808:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761315981.025316:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4223 00000100:00080000:3.0:1761315981.025321:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761315981.025329:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761315981.025343:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761315981.025363:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f364ec74 x1846869307576000/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761315981.025397:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761315981.025400:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761315981.025447:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f364ec74 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307576000:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761315981.025462:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e58cc722 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307575936:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761315981.025506:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761315981.025513:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307575872:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:02020000:0.0:1761315981.028239:0:2262:0:(client.c:1361:ptlrpc_check_status()) 11-0: lustre-OST0000-osc-ffff951bc4d8c800: operation ost_connect to node 192.168.203.151@tcp failed: rc = -19 00000100:00080000:0.0:1761315981.028256:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761315981.028263:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315981.028263:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e93398e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307575872:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315981.028271:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f364ec74 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307576000:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761315981.028284:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000009911ad3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307575808:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761315981.028321:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e58cc722 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307575936:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315981.280810:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315981.346876:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315982.305170:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315982.370150:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315983.328817:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315983.392941:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315984.352816:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315984.416911:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315985.376811:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315985.440930:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761315986.144955:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315986.144965:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761315986.144969:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761315986.144983:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4229 00000100:00080000:2.0:1761315986.144987:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761315986.144993:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761315986.145006:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761315986.145025:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ba5c8e77 x1846869307576256/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761315986.145052:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315986.145055:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761315986.145090:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001426be00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307576128:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315986.145102:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ba5c8e77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307576256:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761315986.145204:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000011b5c99f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307576064:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315986.146797:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ae09c97d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307576192:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761315986.151031:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761315986.151048:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761315986.151085:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ba5c8e77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307576256:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761315986.153948:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001426be00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307576128:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315986.157477:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000011b5c99f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307576064:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315986.160985:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ae09c97d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307576192:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315986.400814:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315986.466040:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315987.428048:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315987.490115:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315988.448741:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315988.513947:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315989.472876:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315989.536870:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315990.500624:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315990.560909:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761315991.264962:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315991.264972:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761315991.264975:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761315991.264990:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4234 00000100:00080000:0.0:1761315991.264994:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761315991.265001:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761315991.265014:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761315991.265035:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000bcaee194 x1846869307576512/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761315991.265062:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761315991.265066:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761315991.265101:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307576384:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315991.265210:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1761315991.265217:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307576320:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315991.265243:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307576448:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761315991.265576:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307576512:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00080000:2.0:1761315991.277055:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761315991.277072:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315991.277084:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307576512:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761315991.277557:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307576384:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315991.277579:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307576320:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761315991.277590:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307576448:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761315991.520830:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315991.584923:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315992.544803:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315992.612738:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315993.568815:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315993.633619:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315994.592942:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315994.657270:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315995.619070:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315995.680908:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761315996.388987:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315996.388994:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761315996.389008:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761315996.389018:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4239 00000100:00080000:2.0:1761315996.389021:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761315996.389027:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761315996.389036:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761315996.389050:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008a641e2e x1846869307576768/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761315996.389070:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761315996.389073:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761315996.389202:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008a641e2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307576768:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761315996.390791:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001426be00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307576576:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315996.393351:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e1be9a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307576640:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315996.393683:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ffb03e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307576704:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761315996.396870:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761315996.396880:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761315996.396890:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008a641e2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307576768:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761315996.397974:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001426be00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307576576:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761315996.398930:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e1be9a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307576640:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761315996.399449:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ffb03e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307576704:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761315996.641414:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315996.707216:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315997.665626:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315997.729714:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315998.688806:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761315998.752851:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761315999.712810:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761315999.778856:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761316000.736845:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316000.801685:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761316001.504832:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761316001.504839:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761316001.504842:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761316001.504852:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4244 00000100:00080000:2.0:1761316001.504856:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761316001.504862:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761316001.504871:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761316001.504885:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000001426be00 x1846869307577024/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761316001.504927:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761316001.504931:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761316001.504963:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e1be9a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307576832:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761316001.505048:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ffb03e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307576960:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761316001.505076:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761316001.505082:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008a641e2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307576896:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761316001.505543:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001426be00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307577024:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761316001.508278:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008a641e2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307576896:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761316001.508296:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e1be9a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307576832:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761316001.508305:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ffb03e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307576960:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761316001.508405:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761316001.508410:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761316001.508413:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001426be00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307577024:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761316001.761478:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316001.825583:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316002.786108:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316002.849054:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316003.811981:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316003.873870:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316004.832779:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316004.896938:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316005.856768:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316005.920902:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316006.625749:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007673f690 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307577088:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761316006.626376:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00080000:3.0:1761316006.626380:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1761316006.626385:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f597cda6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307577216:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761316006.626397:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761316006.626400:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761316006.626404:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4249 00000100:00080000:3.0:1761316006.626406:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761316006.626412:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000100:00100000:2.0:1761316006.626417:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d32bed84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307577152:192.168.203.151@tcp:400:kworker/u8:1.0 00000008:00100000:3.0:1761316006.626422:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761316006.626438:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000101f852a x1846869307577280/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761316006.626464:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761316006.626467:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761316006.626623:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000101f852a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307577280:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761316006.639423:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f597cda6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307577216:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761316006.639441:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d32bed84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307577152:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761316006.639450:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007673f690 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307577088:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761316006.640164:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761316006.640170:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761316006.640175:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000101f852a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307577280:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761316006.882531:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316006.944811:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316007.904774:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316007.970837:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316008.929797:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316008.993362:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316009.952758:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316010.017428:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316010.977251:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316011.045229:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761316011.746366:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761316011.746376:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761316011.746380:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761316011.746394:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4254 00000100:00080000:2.0:1761316011.746398:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761316011.746406:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761316011.746419:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761316011.746438:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ae09c97d x1846869307577536/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761316011.746465:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761316011.746469:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761316011.746507:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e1be9a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307577408:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761316011.746671:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1761316011.746678:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ffb03e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307577344:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761316011.746706:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008a641e2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307577472:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761316011.747058:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ae09c97d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307577536:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761316011.750717:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ffb03e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307577344:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761316011.752709:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008a641e2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307577472:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761316011.752762:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e1be9a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307577408:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761316011.752934:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761316011.752941:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761316011.752946:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ae09c97d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307577536:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761316012.000794:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316012.066377:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316013.025178:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316013.089060:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316014.048793:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316014.118322:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316015.094369:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316015.136874:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316016.096996:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316016.161784:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761316016.865434:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761316016.865441:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761316016.865444:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761316016.865454:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4259 00000100:00080000:2.0:1761316016.865457:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761316016.865462:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761316016.865471:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761316016.865483:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000011b5c99f x1846869307577792/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761316016.865541:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000011b5c99f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307577792:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761316016.865899:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008a641e2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307577664:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761316016.865958:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1761316016.865964:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e1be9a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307577600:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761316016.865988:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ffb03e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307577728:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761316016.866737:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761316016.866742:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00080000:1.0:1761316016.868209:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761316016.868217:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761316016.868225:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000011b5c99f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307577792:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761316016.868478:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e1be9a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307577600:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761316016.868492:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ffb03e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307577728:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761316016.868502:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008a641e2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307577664:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761316017.120758:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316017.185376:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316018.144777:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316018.208876:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316019.170799:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316019.238869:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316020.192771:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316020.260671:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316021.216690:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316021.280836:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761316021.984869:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761316021.984876:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761316021.984879:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761316021.984890:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4264 00000100:00080000:0.0:1761316021.984893:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761316021.984899:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761316021.984908:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761316021.984923:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e00b5e9d x1846869307578048/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761316021.984946:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761316021.984950:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761316021.984973:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307577920:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761316021.985021:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307578048:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761316021.985055:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1761316021.985062:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307577856:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761316021.985084:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307577984:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761316021.995525:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307577920:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761316021.995798:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307577856:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761316021.995826:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307577984:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761316021.996521:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761316021.996528:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761316021.996534:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307578048:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761316022.240773:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316022.304980:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316023.264764:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316023.328943:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316024.288768:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316024.352855:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316025.314273:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316025.377066:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316026.337111:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316026.400880:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761316027.104944:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761316027.104954:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761316027.104959:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761316027.104972:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4270 00000100:00080000:0.0:1761316027.104977:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761316027.104985:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761316027.104998:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761316027.105028:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307578112:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761316027.105093:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000005b900d12 x1846869307578304/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761316027.105120:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761316027.105124:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761316027.105177:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307578240:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761316027.105224:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761316027.105238:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307578304:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761316027.105240:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307578176:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761316027.110037:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761316027.110052:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761316027.110096:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307578304:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761316027.110320:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307578176:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761316027.110344:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307578240:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761316027.111631:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307578112:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761316027.361582:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316027.428357:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316028.384731:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316028.452928:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316029.408788:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316029.472930:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316030.434741:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316030.499073:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316031.456804:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316031.520922:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761316032.224958:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761316032.224966:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761316032.224971:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761316032.224984:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4275 00000100:00080000:2.0:1761316032.224988:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761316032.224995:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761316032.225007:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761316032.225059:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000039978a4e x1846869307578560/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761316032.225084:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761316032.225088:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761316032.225122:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e662bceb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307578432:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761316032.225241:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1761316032.225248:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ba5c8e77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307578368:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761316032.225273:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ee08055d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307578496:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761316032.225370:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039978a4e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307578560:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761316032.230754:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ba5c8e77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307578368:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761316032.232990:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e662bceb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307578432:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761316032.237504:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ee08055d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307578496:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761316032.238797:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761316032.238810:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761316032.238819:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039978a4e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307578560:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761316032.480779:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316032.544925:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316033.504957:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316033.568901:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316034.528809:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316034.595884:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316035.552817:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316035.624239:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316036.576723:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316036.640831:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316037.345320:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307578624:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761316037.345915:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761316037.345925:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307578688:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761316037.346208:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761316037.346214:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00100000:1.0:1761316037.346214:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307578752:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761316037.346217:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761316037.346223:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4280 00000100:00080000:0.0:1761316037.346227:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761316037.346233:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761316037.346248:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761316037.346267:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000fe4020c4 x1846869307578816/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761316037.346281:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761316037.346303:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761316037.346305:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307578816:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00080000:2.0:1761316037.349298:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:0.0:1761316037.349333:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307578752:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761316037.349337:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761316037.349341:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307578816:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761316037.349349:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307578688:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761316037.349359:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307578624:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761316037.604734:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316037.665688:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316038.624831:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316038.688823:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316039.649107:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316039.712954:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316040.674932:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316040.736899:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316041.696777:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316041.763332:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761316042.465861:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761316042.465870:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761316042.465875:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761316042.465887:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4285 00000100:00080000:1.0:1761316042.465891:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761316042.465899:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761316042.465913:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761316042.465932:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000eb8b1f19 x1846869307579072/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761316042.465960:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761316042.465964:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761316042.466054:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307579072:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761316042.466224:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307578944:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761316042.466264:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1761316042.466269:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307578880:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761316042.466293:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307579008:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761316042.476508:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761316042.476524:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761316042.476536:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307579072:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761316042.476742:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307578944:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761316042.476772:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307578880:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761316042.476785:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307579008:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761316042.720801:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316042.784996:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316043.745868:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316043.808929:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316044.769463:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316044.833535:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316045.792799:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316045.857884:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316046.816803:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316046.883049:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761316047.584879:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761316047.585102:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761316047.585106:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761316047.585120:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4290 00000100:00080000:3.0:1761316047.585142:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761316047.585150:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761316047.585163:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761316047.585184:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000039277a44 x1846869307579328/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761316047.585212:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761316047.585215:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761316047.585251:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307579200:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761316047.585285:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307579328:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761316047.585384:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1761316047.585396:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307579136:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761316047.585435:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307579264:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761316047.590985:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307579200:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761316047.593464:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307579136:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761316047.593492:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307579264:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761316047.595093:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761316047.595105:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761316047.595114:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307579328:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761316047.842382:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316047.909598:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316048.864750:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316048.929977:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316049.888967:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316049.953040:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316050.916086:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316050.981559:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316051.936781:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316052.000854:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761316052.706106:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761316052.706113:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761316052.706116:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761316052.706126:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4295 00000100:00080000:2.0:1761316052.706129:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761316052.706135:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761316052.706144:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761316052.706157:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008b3aaddc x1846869307579584/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761316052.706186:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761316052.706189:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761316052.706217:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e662bceb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307579456:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761316052.706232:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b3aaddc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307579584:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761316052.706358:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ee08055d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307579392:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761316052.711479:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ba5c8e77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307579520:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761316052.713487:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e662bceb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307579456:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761316052.713597:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ee08055d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307579392:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761316052.713625:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761316052.713629:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761316052.713684:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b3aaddc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307579584:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761316052.718599:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ba5c8e77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307579520:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761316052.961832:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316053.024940:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316053.984738:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316054.050245:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316055.008805:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316055.072894:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316056.033247:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316056.097799:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316057.056770:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316057.120900:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316057.825288:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307579648:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761316057.825416:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00080000:0.0:1761316057.825421:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1761316057.825422:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307579776:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761316057.825427:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761316057.825431:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1761316057.825452:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307579712:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761316057.825469:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4300 00000100:00080000:0.0:1761316057.825473:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761316057.825479:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761316057.825491:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761316057.825508:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000005a420586 x1846869307579840/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761316057.825538:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761316057.825541:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761316057.826215:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307579840:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00080000:3.0:1761316057.829872:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761316057.829890:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761316057.829904:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307579840:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761316057.832827:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307579776:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761316057.832864:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307579648:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761316057.833207:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307579712:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761316058.080831:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316058.145545:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316059.105603:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316059.169599:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316060.128872:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316060.197310:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316061.152797:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316061.216915:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316062.177026:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316062.241693:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761316062.945026:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761316062.945037:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761316062.945042:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761316062.945056:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4305 00000100:00080000:2.0:1761316062.945061:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761316062.945070:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761316062.945082:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761316062.945098:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006e3466f5 x1846869307580096/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761316062.945124:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761316062.945127:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761316062.945824:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e3466f5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307580096:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761316062.946255:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e662bceb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307579904:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761316062.946308:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004592c0a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307580032:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761316062.946334:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761316062.946339:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000015f165e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307579968:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761316062.958881:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000015f165e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307579968:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761316062.958907:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e662bceb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307579904:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761316062.958918:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004592c0a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307580032:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761316062.963029:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761316062.963047:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761316062.963060:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e3466f5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307580096:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761316063.200853:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316063.269372:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316064.224835:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316064.293925:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316065.248806:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316065.313725:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316066.272791:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316066.337834:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316067.296807:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316067.361606:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761316068.066797:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761316068.066807:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761316068.066810:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761316068.066824:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4311 00000100:00080000:0.0:1761316068.066827:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761316068.066835:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761316068.066848:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761316068.066867:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000005a420586 x1846869307580352/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761316068.066893:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761316068.066897:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761316068.066984:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307580160:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761316068.067097:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307580288:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761316068.067135:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761316068.067141:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307580224:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761316068.067511:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307580352:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761316068.070015:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307580160:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761316068.070349:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307580288:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761316068.070352:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761316068.070359:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761316068.070366:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307580352:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761316068.070395:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307580224:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761316068.323159:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316068.389826:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316069.344847:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316069.409400:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316070.375044:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316070.433894:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316071.393235:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316071.457415:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316072.416806:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316072.483077:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761316073.187441:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761316073.187450:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761316073.187454:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761316073.187467:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4316 00000100:00080000:1.0:1761316073.187471:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761316073.187479:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761316073.187492:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761316073.187510:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000009cec3faa x1846869307580608/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761316073.187537:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761316073.187541:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761316073.187576:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307580480:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761316073.187725:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1761316073.187776:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307580416:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761316073.187804:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307580544:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761316073.191223:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307580608:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761316073.193091:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307580480:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761316073.194410:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761316073.194421:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761316073.194429:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307580608:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761316073.194609:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307580416:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761316073.194622:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307580544:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761316073.445181:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316073.505157:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316074.468526:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316074.529996:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316075.488796:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316075.555042:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316076.513617:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316076.579337:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316077.536764:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316077.600910:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761316078.306532:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761316078.306540:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761316078.306544:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761316078.306554:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4321 00000100:00080000:3.0:1761316078.306557:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761316078.306564:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761316078.306574:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761316078.306590:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000cc6dd8ca x1846869307580864/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761316078.306614:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761316078.306617:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761316078.306667:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307580736:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761316078.306798:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1761316078.306806:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307580672:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761316078.306831:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307580800:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761316078.307441:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cc6dd8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307580864:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761316078.312127:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307580672:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761316078.312149:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307580800:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761316078.312179:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307580736:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761316078.314755:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761316078.314764:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761316078.314772:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc6dd8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307580864:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761316078.560781:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761316078.624846:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316079.584766:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316079.648910:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316080.610848:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316080.675159:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316081.632744:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761316081.697057:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316082.656801:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316082.721874:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761316083.424942:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761316083.424949:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761316083.424952:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761316083.424963:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4326 00000100:00080000:0.0:1761316083.424966:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761316083.424972:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761316083.424981:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761316083.424996:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000546b1bfe x1846869307581120/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761316083.425020:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761316083.425023:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761316083.425051:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307580992:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761316083.425089:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307581120:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761316083.425145:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1761316083.425155:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307580928:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761316083.425182:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307581056:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761316083.429585:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307580992:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761316083.429614:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307580928:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761316083.429624:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307581056:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761316083.430067:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761316083.430072:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761316083.430076:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307581120:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761316083.680817:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316083.746110:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316084.704747:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761316084.769634:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316085.731055:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316085.792981:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316086.752933:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316086.818166:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316087.779819:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761316087.841020:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761316088.545292:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307581184:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761316088.547223:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761316088.547242:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307581248:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761316088.547741:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307581312:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761316088.548078:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761316088.548084:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761316088.548088:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761316088.548094:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4331 00000100:00080000:1.0:1761316088.548097:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761316088.548102:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761316088.548112:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761316088.548133:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000009cec3faa x1846869307581376/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761316088.548148:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761316088.548153:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761316088.548206:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307581376:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761316088.552694:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307581312:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761316088.552723:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307581248:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761316088.552733:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307581184:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761316088.556061:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761316088.556077:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761316088.556091:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307581376:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761316088.800965:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316088.865292:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316089.824849:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316089.888921:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316090.849829:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761316090.913976:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316091.872822:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316091.937421:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316092.896870:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316092.961225:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761316093.668714:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761316093.668725:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761316093.668729:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761316093.668743:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4336 00000100:00080000:3.0:1761316093.668747:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761316093.668755:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761316093.668768:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761316093.668787:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000004e00d332 x1846869307581632/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761316093.668835:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761316093.668838:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761316093.668876:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307581440:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761316093.668971:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307581568:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761316093.669003:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761316093.669007:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307581504:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761316093.671671:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004e00d332 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307581632:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00080000:1.0:1761316093.680486:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:3.0:1761316093.680495:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307581504:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761316093.680498:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761316093.680503:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004e00d332 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307581632:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761316093.680518:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307581440:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761316093.680528:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307581568:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761316093.920809:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761316093.984880:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316094.944810:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316095.010848:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316095.968793:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316096.032836:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316096.993106:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761316097.059670:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316098.016782:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316098.083767:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761316098.787693:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761316098.787725:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761316098.787730:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761316098.787744:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4341 00000100:00080000:0.0:1761316098.787749:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761316098.787758:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761316098.787771:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761316098.787792:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007a68d9a8 x1846869307581888/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761316098.787817:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761316098.787820:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761316098.788767:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307581760:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761316098.789095:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307581696:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761316098.790838:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307581888:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761316098.790964:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307581824:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761316098.804953:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761316098.804969:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761316098.804981:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307581888:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761316098.807413:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307581696:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761316098.807442:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307581824:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761316098.807496:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307581760:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761316099.040760:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316099.104937:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316100.064758:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761316100.128885:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316101.089986:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316101.152845:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316102.113429:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316102.177708:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316103.138359:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761316103.201747:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761316103.905362:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761316103.905371:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761316103.905375:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761316103.905392:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4346 00000100:00080000:2.0:1761316103.905397:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761316103.905405:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761316103.905418:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761316103.905438:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000039eebc2 x1846869307582144/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761316103.905465:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761316103.905490:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761316103.905530:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000039eebc2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307582144:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761316103.905552:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b2298797 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307581952:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761316103.905574:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eca34e1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307582016:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761316103.905691:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e9376149 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307582080:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761316103.909978:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b2298797 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307581952:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761316103.911807:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761316103.911821:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761316103.911832:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000039eebc2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307582144:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761316103.917225:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e9376149 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307582080:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761316103.917377:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eca34e1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307582016:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761316104.161139:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316104.225336:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316105.184821:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316105.248967:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316106.208799:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761316106.273351:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316107.234603:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316107.299287:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316108.260044:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316108.321444:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761316109.024835:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761316109.024842:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761316109.024845:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761316109.024855:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4351 00000100:00080000:2.0:1761316109.024859:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761316109.024866:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761316109.024875:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761316109.024889:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f220504f x1846869307582400/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761316109.024959:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f220504f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307582400:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00080000:2.0:1761316109.025383:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761316109.025387:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761316109.025411:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b2298797 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307582272:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761316109.025448:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1761316109.025454:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e9376149 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307582208:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761316109.025475:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307582336:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761316109.035817:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761316109.035828:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761316109.035837:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f220504f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307582400:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761316109.036474:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e9376149 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307582208:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761316109.036501:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307582336:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761316109.036512:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b2298797 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307582272:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761316109.281544:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761316109.344958:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316110.304796:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316110.370007:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316111.330573:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316111.392836:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316112.352765:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761316112.418135:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316113.377198:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316113.440875:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761316114.146183:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761316114.146190:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761316114.146193:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761316114.146204:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4357 00000100:00080000:3.0:1761316114.146207:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761316114.146213:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761316114.146221:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761316114.146235:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006c8fcfe5 x1846869307582656/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761316114.146261:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761316114.146264:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761316114.146288:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307582528:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761316114.146347:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307582464:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761316114.146576:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307582656:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761316114.148076:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307582592:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761316114.152286:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307582528:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761316114.152312:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761316114.152318:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761316114.152322:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307582656:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761316114.152432:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307582464:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761316114.152467:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307582592:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761316114.400813:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316114.465788:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316115.424766:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761316115.489887:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316116.451305:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316116.513774:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316117.473457:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316117.536823:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316118.496769:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761316118.560901:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761316119.265021:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761316119.265031:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761316119.265035:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761316119.265079:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4362 00000100:00080000:1.0:1761316119.265083:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00100000:0.0:1761316119.265089:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307582720:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761316119.265097:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761316119.265115:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761316119.265141:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000003871442b x1846869307582912/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761316119.265167:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761316119.265171:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761316119.265208:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307582848:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761316119.265226:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307582784:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761316119.265740:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307582912:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761316119.271032:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307582848:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761316119.271230:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307582784:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761316119.278827:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761316119.278843:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761316119.278855:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307582912:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761316119.279213:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307582720:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761316119.521031:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316119.585688:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316120.544837:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316120.608904:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316121.569529:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761316121.636599:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316122.592809:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316122.656835:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316123.616802:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316123.681943:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761316124.384915:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761316124.384923:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761316124.384926:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761316124.384938:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4367 00000100:00080000:1.0:1761316124.384941:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761316124.384947:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761316124.384957:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761316124.384973:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a1719d14 x1846869307583168/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761316124.384995:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761316124.384998:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761316124.385025:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307583040:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761316124.385088:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307583168:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761316124.385125:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307582976:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761316124.385174:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307583104:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761316124.395718:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307583040:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761316124.395855:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761316124.395861:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761316124.395866:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307583168:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761316124.395960:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307582976:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761316124.395975:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307583104:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761316124.640955:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761316124.706103:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316125.664771:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316125.730903:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316126.689307:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316126.752972:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316127.712960:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316127.778941:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316128.736771:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316128.801600:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761316129.505118:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761316129.505126:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761316129.505129:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761316129.505141:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4372 00000100:00080000:3.0:1761316129.505144:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761316129.505150:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761316129.505159:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761316129.505174:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000bfffbafe x1846869307583424/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761316129.505197:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761316129.505201:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761316129.505286:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307583232:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761316129.505415:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307583360:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761316129.505449:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761316129.505455:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f133bb2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307583296:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761316129.505776:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307583424:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761316129.509764:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307583232:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761316129.515890:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f133bb2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307583296:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761316129.515909:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307583360:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761316129.519774:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761316129.519789:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761316129.519799:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307583424:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761316129.762463:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316129.825396:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316130.784773:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316130.849959:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316131.809306:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316131.874066:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316132.832764:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316132.897004:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316133.856786:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316133.920962:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761316134.626210:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761316134.626216:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761316134.626219:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761316134.626230:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4377 00000100:00080000:1.0:1761316134.626247:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761316134.626252:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761316134.626261:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761316134.626290:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a1719d14 x1846869307583680/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761316134.626338:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761316134.626342:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761316134.626370:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307583552:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761316134.626452:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1761316134.626459:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307583488:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761316134.626481:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307583616:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761316134.628006:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307583680:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761316134.632176:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307583488:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761316134.637580:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307583616:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761316134.637605:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307583552:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:02020000:3.0:1761316134.638886:0:2262:0:(client.c:1361:ptlrpc_check_status()) 11-0: lustre-OST0000-osc-ffff951bc4d8c800: operation ost_connect to node 192.168.203.151@tcp failed: rc = -19 00000100:00080000:3.0:1761316134.638896:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761316134.638900:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761316134.638904:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1719d14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307583680:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761316134.880773:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316134.945262:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316135.904762:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316135.968902:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316136.930868:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316136.992935:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316137.952733:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316138.018083:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316138.976782:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316139.042127:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761316139.745766:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761316139.745774:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761316139.745777:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761316139.745787:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4382 00000100:00080000:3.0:1761316139.745791:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761316139.745798:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761316139.745811:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761316139.745833:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000cdcc674d x1846869307583936/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761316139.745860:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761316139.745863:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761316139.745956:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307583744:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761316139.746050:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307583872:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761316139.746098:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761316139.746104:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307583808:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761316139.747139:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cdcc674d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307583936:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761316139.750999:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307583744:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761316139.764981:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307583808:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761316139.765007:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307583872:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761316139.766528:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761316139.766538:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761316139.766549:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cdcc674d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307583936:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761316140.001058:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316140.069462:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316141.024946:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316141.088860:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316142.048807:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316142.112868:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316143.073597:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316143.136902:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316144.097391:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316144.160913:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761316144.864938:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761316144.864947:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761316144.864951:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761316144.864964:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4387 00000100:00080000:2.0:1761316144.864968:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761316144.864975:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761316144.864987:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761316144.865006:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b4eac89a x1846869307584192/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761316144.865032:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761316144.865035:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761316144.865091:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b4eac89a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307584192:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761316144.865135:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307584064:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761316144.865220:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1761316144.865229:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b2298797 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307584000:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761316144.865267:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e9376149 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307584128:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761316144.871096:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b2298797 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307584000:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761316144.871543:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e9376149 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307584128:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761316144.871560:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307584064:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761316144.871769:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761316144.871775:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761316144.871781:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b4eac89a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307584192:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761316145.120778:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316145.186178:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316146.144708:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316146.209013:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316147.169858:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761316147.233304:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316148.192846:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316148.257401:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316149.217094:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316149.281565:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761316149.984970:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761316149.984979:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761316149.984983:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761316149.985021:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4392 00000100:00080000:1.0:1761316149.985026:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761316149.985034:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761316149.985047:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761316149.985068:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a15ef099 x1846869307584448/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761316149.985094:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761316149.985098:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761316149.985469:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307584256:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761316149.985670:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307584448:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761316149.986089:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307584384:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761316149.986132:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761316149.986140:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307584320:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761316149.989032:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307584320:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761316149.989057:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307584384:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761316149.989675:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761316149.989683:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761316149.989690:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a15ef099 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307584448:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761316149.990142:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000163e3801 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307584256:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761316150.240800:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761316150.305975:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316151.264770:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316151.328985:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316152.288973:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316152.352911:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316153.312877:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761316153.378269:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316154.337095:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316154.400890:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761316155.104887:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761316155.104894:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761316155.104897:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761316155.104909:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4398 00000100:00080000:2.0:1761316155.104912:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761316155.104919:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761316155.104928:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761316155.104944:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aea3302e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307584512:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761316155.104949:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b4eac89a x1846869307584704/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761316155.105034:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b4eac89a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307584704:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761316155.105084:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002292ab57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307584640:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761316155.105144:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761316155.105151:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000007136630 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307584576:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761316155.105858:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761316155.105864:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761316155.109894:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000007136630 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307584576:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761316155.110075:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761316155.110080:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761316155.110086:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b4eac89a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307584704:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761316155.110217:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002292ab57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307584640:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761316155.110238:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aea3302e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307584512:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761316155.360757:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316155.425582:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316156.385051:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761316156.449713:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316157.408771:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316157.472867:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316158.432831:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316158.497458:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316159.456728:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761316159.522205:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316160.225273:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307584768:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761316160.225955:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761316160.225964:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307584832:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761316160.226098:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761316160.226103:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761316160.226107:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761316160.226113:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4403 00000100:00080000:3.0:1761316160.226116:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761316160.226145:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761316160.226156:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761316160.226172:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000038534f0b x1846869307584960/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761316160.226188:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761316160.226191:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761316160.226209:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761316160.226218:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307584896:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761316160.226488:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000038534f0b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307584960:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761316160.233656:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307584768:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761316160.237526:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307584896:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761316160.237874:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761316160.237885:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761316160.237893:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000038534f0b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307584960:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761316160.240103:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307584832:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761316160.480778:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316160.545025:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316161.504923:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316161.569064:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316162.528749:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761316162.592911:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316163.553014:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316163.617004:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316164.577426:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316164.641857:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761316165.344899:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761316165.344908:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761316165.344913:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761316165.344926:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4408 00000100:00080000:3.0:1761316165.344930:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761316165.344937:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000100:00100000:2.0:1761316165.344943:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307585024:192.168.203.151@tcp:400:kworker/u8:1.0 00000008:00100000:3.0:1761316165.344949:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761316165.344970:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000038534f0b x1846869307585216/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761316165.345001:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761316165.345004:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761316165.345063:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000626f98db pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307585088:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761316165.345082:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307585152:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761316165.345353:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000038534f0b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307585216:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761316165.348740:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000626f98db pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307585088:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761316165.348884:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761316165.348891:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761316165.348895:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000038534f0b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307585216:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761316165.348924:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000d82b82f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307585024:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761316165.348935:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ddd3c50a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307585152:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761316165.600803:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761316165.669208:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316166.624801:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316166.689044:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316167.649627:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316167.713482:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316168.672715:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761316168.737926:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316169.697351:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316169.761269:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761316170.467475:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761316170.467486:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761316170.467491:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1761316170.467498:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307585280:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761316170.467509:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4413 00000100:00080000:0.0:1761316170.467513:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761316170.467520:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761316170.467546:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761316170.467567:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000057b18867 x1846869307585472/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761316170.467609:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307585408:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761316170.467663:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761316170.467668:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761316170.467676:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761316170.467683:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307585344:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761316170.467999:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307585472:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761316170.473600:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307585280:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761316170.473634:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307585408:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761316170.474465:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307585344:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761316170.475050:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761316170.475064:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761316170.475071:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307585472:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761316170.720831:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316170.787754:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316171.744779:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761316171.809410:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316172.768800:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316172.835898:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316173.792819:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316173.856941:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316174.816740:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761316174.881633:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761316175.585848:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761316175.585858:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761316175.585862:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761316175.585875:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4418 00000100:00080000:3.0:1761316175.585879:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761316175.585885:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761316175.585896:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761316175.585913:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e6d9e11e x1846869307585728/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761316175.585939:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761316175.585942:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761316175.586039:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307585728:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761316175.586232:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000626f98db pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307585536:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761316175.586265:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cdcc674d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307585664:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761316175.586286:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761316175.586290:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307585600:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761316175.591159:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000602b09d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307585600:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761316175.591179:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000626f98db pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307585536:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761316175.591188:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cdcc674d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307585664:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761316175.599861:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761316175.599875:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761316175.599886:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e6d9e11e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307585728:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761316175.840809:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316175.905896:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316176.864841:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316176.929718:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316177.892162:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761316177.953746:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316178.913043:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316178.979383:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316179.937794:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316180.002020:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761316180.705068:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761316180.705078:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761316180.705082:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761316180.705095:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4423 00000100:00080000:0.0:1761316180.705099:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761316180.705106:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761316180.705119:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761316180.705139:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000bcaee194 x1846869307585984/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761316180.705172:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761316180.705175:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761316180.705226:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307585856:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761316180.705312:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307585792:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761316180.705355:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307585920:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761316180.706039:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307585984:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761316180.714874:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307585856:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761316180.714875:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307585920:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761316180.717908:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761316180.717924:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761316180.717937:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307585984:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761316180.717943:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307585792:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761316180.963142:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761316181.025785:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316181.985734:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316182.048945:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316183.009215:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316183.073142:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316184.032939:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761316184.099512:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316185.056811:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316185.121434:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761316185.824894:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761316185.824903:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761316185.824907:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761316185.824921:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4428 00000100:00080000:0.0:1761316185.824925:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761316185.824933:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761316185.824945:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761316185.824993:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000034a6cd79 x1846869307586240/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761316185.825019:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761316185.825023:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761316185.825161:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307586112:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761316185.825288:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1761316185.825294:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307586048:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761316185.825321:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307586176:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761316185.825800:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307586240:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761316185.833379:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008e59d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307586048:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761316185.833414:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bcaee194 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307586176:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761316185.834744:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307586112:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761316185.834910:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761316185.834916:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761316185.834921:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307586240:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761316186.082123:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316186.145363:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316187.105419:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761316187.170124:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316188.128690:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316188.193052:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316189.152770:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316189.218032:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316190.176803:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761316190.241031:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761316190.945171:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761316190.945180:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761316190.945203:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761316190.945216:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4433 00000100:00080000:1.0:1761316190.945220:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761316190.945227:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761316190.945241:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761316190.945272:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000540c3ce6 x1846869307586496/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761316190.945302:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761316190.945306:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761316190.945341:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307586368:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761316190.945445:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1761316190.945451:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307586304:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761316190.945476:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307586432:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761316190.946070:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307586496:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761316190.948140:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307586368:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761316190.950455:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761316190.950468:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761316190.950480:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000540c3ce6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307586496:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761316190.950577:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307586304:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761316190.950592:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307586432:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761316191.201466:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316191.264916:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316192.224816:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316192.290820:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316193.248771:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761316193.312910:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316194.273513:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316194.337519:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316195.296765:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316195.360934:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761316196.064854:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761316196.064861:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761316196.064864:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761316196.064873:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4439 00000100:00080000:1.0:1761316196.064900:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761316196.064918:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:1.0:1761316196.064927:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:1.0:1761316196.064940:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000009cec3faa x1846869307586752/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761316196.064960:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761316196.064963:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761316196.064985:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307586624:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761316196.065059:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1761316196.065063:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307586560:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761316196.065078:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307586688:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761316196.065176:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307586752:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761316196.068238:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb8b1f19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307586688:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761316196.068253:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed2f9bca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307586624:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761316196.068635:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761316196.068674:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761316196.068679:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cec3faa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307586752:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761316196.070910:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003871442b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307586560:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761316196.321579:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761316196.388381:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316197.344760:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316197.408885:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316198.368787:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316198.432833:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316199.392792:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761316199.456888:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316200.418108:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316200.480879:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761316201.184893:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761316201.184900:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761316201.184905:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761316201.184915:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4444 00000100:00080000:0.0:1761316201.184918:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761316201.184924:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761316201.184933:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761316201.184945:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000005b900d12 x1846869307587008/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761316201.184969:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761316201.184972:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761316201.184994:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307586880:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761316201.185073:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1761316201.185079:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307586816:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761316201.185103:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307586944:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761316201.185538:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307587008:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761316201.190117:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307586944:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761316201.190834:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307586880:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761316201.190862:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307586816:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761316201.193968:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761316201.193980:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761316201.193991:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307587008:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761316201.440775:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316201.506290:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316202.464992:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761316202.528980:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316203.488809:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316203.553610:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316204.512769:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316204.578925:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316205.536841:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761316205.600846:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761316206.305567:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761316206.305576:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761316206.305581:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761316206.305594:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4449 00000100:00080000:0.0:1761316206.305598:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761316206.305623:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761316206.305635:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761316206.305684:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000fe4020c4 x1846869307587264/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761316206.305703:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761316206.305706:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761316206.305752:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307587264:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761316206.306380:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307587136:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761316206.306466:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1761316206.306478:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307587072:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761316206.306515:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307587200:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761316206.312775:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307587200:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761316206.312794:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307587136:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761316206.316231:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761316206.316240:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761316206.316249:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307587264:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761316206.316357:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307587072:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761316206.560753:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316206.625905:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316207.584816:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316207.649097:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316208.609668:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761316208.673052:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316209.635897:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316209.696879:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316210.656751:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316210.720993:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761316211.426107:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761316211.426118:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761316211.426122:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761316211.426136:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4454 00000100:00080000:0.0:1761316211.426140:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761316211.426147:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761316211.426160:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761316211.426179:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ace4d025 x1846869307587520/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761316211.426207:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761316211.426210:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761316211.426237:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307587392:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761316211.426342:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1761316211.426349:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307587328:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761316211.426394:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307587456:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761316211.426934:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307587520:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761316211.433162:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307587328:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761316211.433185:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307587456:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761316211.433195:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307587392:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761316211.436197:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761316211.436213:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761316211.436226:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307587520:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761316211.681777:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761316211.747251:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316212.709206:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316212.768877:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316213.729469:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316213.793366:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316214.752784:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761316214.817231:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316215.776723:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316215.844134:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761316216.546080:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761316216.546088:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761316216.546091:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761316216.546102:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4459 00000100:00080000:2.0:1761316216.546105:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761316216.546111:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761316216.546129:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761316216.546145:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b2298797 x1846869307587776/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761316216.546226:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b2298797 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307587776:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00080000:2.0:1761316216.546375:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761316216.546384:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761316216.546413:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307587648:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761316216.546521:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1761316216.546528:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b4eac89a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307587584:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761316216.546554:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e9376149 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307587712:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761316216.552331:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761316216.552341:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761316216.552350:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b2298797 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307587776:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761316216.552452:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e9376149 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307587712:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761316216.552465:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307587648:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761316216.552772:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b4eac89a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307587584:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761316216.800901:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316216.865253:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316217.825150:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316217.891242:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316218.849703:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316218.912980:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316219.873081:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316219.936932:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316220.896757:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316220.960939:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316221.666359:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307587840:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761316221.667770:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307587904:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761316221.668215:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761316221.668223:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307587968:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761316221.668545:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761316221.668553:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761316221.668557:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761316221.668564:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4464 00000100:00080000:0.0:1761316221.668568:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761316221.668575:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761316221.668589:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761316221.668621:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000005a420586 x1846869307588032/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761316221.668737:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307588032:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00080000:0.0:1761316221.669155:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761316221.669163:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00080000:2.0:1761316221.717507:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761316221.717524:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761316221.717537:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005a420586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307588032:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761316221.718617:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307587968:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761316221.718675:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307587904:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761316221.718727:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307587840:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761316221.921193:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316221.986756:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316222.944949:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316223.009703:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316223.968987:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316224.032910:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316224.993835:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316225.058095:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316226.016804:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316226.081376:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761316226.787784:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761316226.787792:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761316226.787796:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761316226.787807:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4469 00000100:00080000:2.0:1761316226.787810:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761316226.787817:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761316226.787825:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761316226.787839:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f220504f x1846869307588288/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761316226.787925:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f220504f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307588288:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761316226.789182:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b4eac89a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307588096:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761316226.789248:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e9376149 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307588224:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761316226.789281:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761316226.789287:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307588160:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761316226.789717:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761316226.789723:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761316226.794105:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b4eac89a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307588096:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761316226.794126:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e9376149 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307588224:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761316226.796776:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761316226.796785:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761316226.796794:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f220504f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307588288:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761316226.797175:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b8c05a37 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307588160:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761316227.041759:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316227.104926:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316228.064772:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316228.129143:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316229.088770:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316229.152920:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316230.112792:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316230.176897:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316231.136763:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316231.200837:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761316231.909691:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761316231.909700:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761316231.909703:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761316231.909714:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4474 00000100:00080000:0.0:1761316231.909717:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761316231.909723:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761316231.909734:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761316231.909747:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000094820351 x1846869307588544/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761316231.909773:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761316231.909776:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761316231.909823:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307588416:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761316231.909915:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1761316231.909922:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307588352:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761316231.909946:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307588480:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761316231.910237:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307588544:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761316231.912622:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ace4d025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307588416:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761316231.916937:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a24c5c6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307588352:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761316231.917164:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000028de4b9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307588480:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761316231.917902:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761316231.917923:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:2.0:1761316231.917929:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094820351 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307588544:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761316232.160799:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316232.225313:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316233.184809:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316233.249335:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316234.208779:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316234.272878:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316235.232764:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316235.297272:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316236.256766:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316236.320962:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761316237.024918:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761316237.024945:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761316237.024950:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761316237.024963:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4479 00000100:00080000:0.0:1761316237.024968:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 6s 00000100:00080000:0.0:1761316237.024975:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761316237.024987:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761316237.025008:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000084f3d917 x1846869307588800/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761316237.025042:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761316237.025046:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761316237.025101:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307588672:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761316237.025207:0:2263:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1761316237.025219:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307588608:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761316237.025247:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307588736:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761316237.027477:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307588800:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761316237.030935:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307588608:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761316237.030963:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307588736:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761316237.030975:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307588672:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761316237.037789:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761316237.037805:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761316237.037818:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084f3d917 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307588800:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761316237.280784:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316237.344916:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316238.306204:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316238.368910:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316239.328837:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316239.393447:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316240.352765:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316240.416883:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316241.376764:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761316241.440986:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761316242.144843:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761316242.144864:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761316242.144867:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761316242.144879:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4485 00000100:00100000:1.0:1761316242.144881:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307588864:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761316242.144885:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 11s 00000100:00080000:0.0:1761316242.144894:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761316242.144904:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761316242.144922:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000005b900d12 x1846869307589056/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761316242.144942:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761316242.144946:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761316242.144981:0:2263:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307588992:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761316242.145007:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307588928:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761316242.145776:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307589056:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761316242.148533:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2263:1846869307588992:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761316242.148533:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000546b1bfe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307588928:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761316242.151373:0:2263:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fe4020c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2263:1846869307588864:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761316242.152203:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761316242.152210:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761316242.152217:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005b900d12 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307589056:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761316242.400764:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316242.465662:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316243.424766:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316243.489244:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316244.448792:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761316244.513082:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316245.473022:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316245.539907:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316246.496874:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316246.560885:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761316247.265187:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761316247.265196:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761316247.265201:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761316247.265215:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4490 00000100:00080000:2.0:1761316247.265221:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 16s 00000100:00080000:2.0:1761316247.265228:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:2.0:1761316247.265240:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761316247.265258:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000004592c0a5 x1846869307589312/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761316247.265325:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004592c0a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307589312:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00080000:2.0:1761316247.267917:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761316247.267928:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761316247.268110:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eca34e1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307589120:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761316247.269386:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000039eebc2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307589184:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761316247.273263:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e3466f5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307589248:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761316247.274399:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761316247.274731:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761316247.274740:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004592c0a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307589312:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761316247.277085:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000039eebc2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307589184:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761316247.277574:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e3466f5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307589248:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761316247.279811:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eca34e1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307589120:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761316247.521177:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761316247.585233:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316248.544781:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316248.609388:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316249.568892:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316249.633822:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761316250.592770:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761316250.656907:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316251.624391:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316251.680988:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761316252.384884:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761316252.384892:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761316252.384895:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761316252.384907:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4495 00000100:00080000:3.0:1761316252.384911:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761316252.384916:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761316252.384926:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761316252.384941:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000bc14d8e x1846869307589568/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761316252.384966:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761316252.384970:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761316252.384995:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307589440:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761316252.385090:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1761316252.385097:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307589376:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761316252.385123:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f133bb2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307589504:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761316252.385873:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307589568:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761316252.388349:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307589376:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761316252.388385:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f133bb2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307589504:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761316252.388395:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307589440:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761316252.391145:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761316252.391154:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:0.0:1761316252.391174:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000bc14d8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307589568:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761316252.640761:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316252.705058:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316253.664755:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761316253.728875:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316254.689499:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316254.753958:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316255.718035:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316255.776810:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316256.736888:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761316256.801662:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761316257.505028:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761316257.505038:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761316257.505043:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761316257.505057:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4500 00000100:00080000:0.0:1761316257.505061:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761316257.505069:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761316257.505084:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761316257.505104:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b5a39135 x1846869307589824/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1761316257.505169:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307589824:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00080000:0.0:1761316257.505282:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761316257.505287:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761316257.505376:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307589632:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761316257.505410:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307589760:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761316257.505443:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761316257.505459:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307589696:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761316257.512958:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761316257.512977:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761316257.512991:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b5a39135 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307589824:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761316257.513104:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307589632:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761316257.513122:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307589760:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761316257.513144:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307589696:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761316257.760864:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316257.835673:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316258.786039:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316258.849774:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316259.808795:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761316259.873072:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316260.832829:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316260.897334:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316261.856833:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316261.922556:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316262.625564:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307589888:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761316262.630746:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761316262.630757:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761316262.630761:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761316262.630774:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4505 00000100:00080000:3.0:1761316262.630778:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1761316262.630874:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f133bb2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307589952:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761316262.631015:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761316262.631027:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761316262.631044:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000004505177d x1846869307590080/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761316262.631059:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761316262.631062:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761316262.631102:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307590080:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761316262.631187:0:2266:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761316262.631193:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307590016:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761316262.637327:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307589888:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761316262.642616:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bfffbafe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307590016:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761316262.642669:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f133bb2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307589952:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761316262.642767:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761316262.642773:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761316262.642778:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307590080:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761316262.882475:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:3.0:1761316262.944874:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316263.904797:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316263.971295:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316264.929851:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316264.992848:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316265.953235:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316266.021349:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316266.977042:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316267.040941:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761316267.744845:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761316267.744863:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761316267.744866:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761316267.744879:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4510 00000100:00080000:3.0:1761316267.744883:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761316267.744890:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761316267.744902:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:2.0:1761316267.744923:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307590144:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761316267.744925:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006c8fcfe5 x1846869307590336/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761316267.744964:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761316267.744969:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761316267.745013:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307590208:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761316267.745226:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307590336:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761316267.746203:0:2266:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307590272:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761316267.749398:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761316267.749409:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761316267.749418:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006c8fcfe5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307590336:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761316267.749497:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307590208:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761316267.749594:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004505177d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2266:1846869307590144:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761316267.749621:0:2266:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004c6aebc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2266:1846869307590272:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761316268.000827:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316268.067140:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316269.024798:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316269.089018:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316270.048770:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316270.112908:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316271.072766:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316271.136836:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316272.096810:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316272.160886:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761316272.865880:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761316272.865890:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761316272.865894:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761316272.865908:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4515 00000100:00080000:0.0:1761316272.865912:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761316272.865920:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761316272.865931:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761316272.865950:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000034a6cd79 x1846869307590592/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761316272.865974:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761316272.865977:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761316272.866040:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307590464:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761316272.866156:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1761316272.866163:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307590400:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761316272.866188:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307590528:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761316272.868061:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307590592:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761316272.873736:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307590400:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761316272.873775:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307590528:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761316272.873793:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307590464:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761316272.875952:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761316272.875967:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761316272.875994:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000034a6cd79 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307590592:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761316273.120940:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316273.185233:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316274.144759:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316274.208881:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316275.168993:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316275.233053:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316276.192813:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316276.257600:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316277.216931:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316277.282777:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761316277.986544:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761316277.986555:0:21016:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761316277.986559:0:21016:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761316277.986573:0:21016:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4520 00000100:00080000:3.0:1761316277.986577:0:21016:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761316277.986584:0:21016:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:3.0:1761316277.986597:0:21016:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:3.0:1761316277.986615:0:21016:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000039277a44 x1846869307590848/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761316277.986710:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307590848:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00080000:3.0:1761316277.986871:0:21016:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761316277.986876:0:21016:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761316277.988856:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307590656:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761316277.988979:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cc6dd8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307590784:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761316277.989014:0:2265:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761316277.989020:0:2265:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004e00d332 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307590720:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761316277.995721:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761316277.995737:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:1.0:1761316277.995751:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039277a44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307590848:192.168.203.151@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761316277.995973:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004e00d332 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307590720:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761316277.995987:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a183f7ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2265:1846869307590656:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761316277.995998:0:2265:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc6dd8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2265:1846869307590784:192.168.203.151@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761316278.241570:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316278.305083:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316279.265369:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316279.329614:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316280.288855:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316280.353349:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316281.312787:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316281.378065:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316282.336750:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316282.401713:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761316283.105492:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761316283.105499:0:38:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761316283.105502:0:38:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761316283.105513:0:38:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: connect to NID 192.168.203.151@tcp last attempt 4526 00000100:00080000:0.0:1761316283.105517:0:38:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761316283.105523:0:38:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff951bc4d8c800: import 000000009e982beb using connection 192.168.203.151@tcp/192.168.203.151@tcp 00000008:00100000:0.0:1761316283.105532:0:38:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 295600128, lost: 0. 00000100:00100000:0.0:1761316283.105549:0:38:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007dcceb4a x1846869307591104/t0(0) o8->lustre-OST0000-osc-ffff951bc4d8c800@192.168.203.151@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761316283.105574:0:38:0:(pinger.c:251:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761316283.105578:0:38:0:(pinger.c:265:ptlrpc_pinger_process_import()) 552c6ffa-be0b-4ca9-86e3-4892e1432c16->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761316283.106589:0:2262:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307591104:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761316283.106792:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307590912:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761316283.106827:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307591040:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761316283.106852:0:2264:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761316283.106872:0:2264:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307590976:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761316283.114131:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e00b5e9d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:803c8c07-d6b3-4fdb-a7b3-4227400366d7:2264:1846869307590912:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761316283.114160:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057b18867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307591040:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761316283.114190:0:2264:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a68d9a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2264:1846869307590976:192.168.203.151@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761316283.114250:0:2262:0:(import.c:86:import_set_state_nolock()) 000000009e982beb lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761316283.114256:0:2262:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.203.151@tcp failed (-19) 00000100:00100000:3.0:1761316283.114261:0:2262:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007dcceb4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:552c6ffa-be0b-4ca9-86e3-4892e1432c16:2262:1846869307591104:192.168.203.151@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761316283.360819:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316283.425387:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316284.385115:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316284.448795:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316285.409340:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316285.473940:0:11248:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316286.432783:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:1.0:1761316286.498635:0:5579:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761316287.461531:0:20811:0:(client.c:2500:ptlrpc_set_wait()) set 000000003bbf2145 going to sleep for 0 seconds 00010000:00010000:0.0:1761316287.520877:0:5578:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) Debug log: 22154 lines, 22154 kept, 0 dropped, 0 bad.